Video or image coding based on mapping of luma samples and scaling of chroma samples

ABSTRACT

According to the disclosure of the present document, the granularity of value(s) of a chroma residual scaling factor can be controlled through explicit signaling of information on the value(s) of the chroma residual scaling factor. In addition, chroma residual scaling can be efficiently performed through implicit derivation of the information on the value(s) of the chroma residual scaling factor.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The technique of the present document is related to video or imagecoding based mapping of luma samples and scaling chroma samples.

Related Art

Recently, demand for high-resolution, high-quality image/video such as4K or 8K or higher ultra high definition (UHD) image/video has increasedin various fields. As image/video data has high resolution and highquality, the amount of information or bits to be transmitted increasesrelative to the existing image/video data, and thus, transmitting imagedata using a medium such as an existing wired/wireless broadband line oran existing storage medium or storing image/video data using existingstorage medium increase transmission cost and storage cost.

In addition, interest and demand for immersive media such as virtualreality (VR) and artificial reality (AR) content or holograms hasrecently increased and broadcasting for image/video is havingcharacteristics different from reality images such as game images hasincreased.

Accordingly, a highly efficient image/video compression technology isrequired to effectively compress, transmit, store, and reproduceinformation of a high-resolution, high-quality image/video havingvarious characteristics as described above.

Further, in order to improve compression efficiency and increasesubjective/objective visual qualities, a luma mapping with chromascaling (LMCS) procedure is performed and there are discusses aboutplans for increasing a subjective/objective image quality through theLMCS procedure.

SUMMARY

According to an embodiment of the present document, a method and anapparatus for increasing image coding efficiency are provided.

According to an embodiment of the present document, efficient filteringapplication method and apparatus are provided.

According to an embodiment of the present document, efficient LMCSapplication method and apparatus are provided.

According to an embodiment of the present document, the LMCS codewords(or a range thereof) may be constrained.

According to an embodiment of the present document, a single chromaresidual scaling factor directly signaled in chroma scaling of LMCS maybe used.

According to an embodiment of the present document, linear mapping(linear LMCS) may be used.

According to an embodiment of the present document, information on pivotpoints required for linear mapping may be explicitly signaled.

According to an embodiment of the present document, a flexible number ofbins may be used for luma mapping.

According to an embodiment of the present document, an index derivationprocedure for inverse luma mapping and/or chroma residual scaling may besimplified.

According to an embodiment of the present document, the LMCS proceduremay be applied even when luma and chroma blocks in one coding tree unit(CTU) have a separate block tree structure (dual tree structure).

According to an embodiment of the present document, information about(a) value(s) of a chroma residual scaling factor may be explicitlysignaled or implicitly derived.

According to an embodiment of the present document, a video/imagedecoding method performed by a decoding apparatus is provided.

According to an embodiment of the present document, a decoding apparatusfor performing video/image decoding is provided.

According to an embodiment of the present document, a video/imageencoding method performed by an encoding apparatus is provided.

According to an embodiment of the present document, an encodingapparatus for performing video/image encoding is provided.

According to one embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded video/imageinformation, generated according to the video/image encoding methoddisclosed in at least one of the embodiments of the present document, isstored.

According to an embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded information orencoded video/image information, causing to perform the video/imagedecoding method disclosed in at least one of the embodiments of thepresent document by the decoding apparatus, is stored.

Advantageous Effects

According to an embodiment of the present document, overall image/videocompression efficiency may be improved.

According to an embodiment of the present document, subjective/objectivevisual quality may be improved through efficient filtering.

According to an embodiment of the present document, the LMCS process forimage/video coding may be efficiently performed.

According to an embodiment of the present document, it is possible tominimize resources/costs (of software or hardware) required for the LMCSprocess.

According to an embodiment of the present document, hardwareimplementation for the LMCS process may be facilitated.

According to an embodiment of the present document, a division operationrequired for derivation of LMCS codewords in mapping (reshaping) can beremoved or minimized by constraint of the LMCS codewords (or rangethereof).

According to an embodiment of the present document, latency according topiecewise index identification may be removed by using a single chromaresidual scaling factor.

According to an embodiment of the present document, a chroma residualscaling process can be performed without depending on (reconstructionof) a luma block by using the linear mapping in LMCS, and thus latencyin scaling can be removed.

According to an embodiment of the present document, mapping efficiencyin LMCS may be increased.

According to an embodiment of the present document, the complexity ofthe LMCS may be reduced through simplification of an index derivationprocedure for inverse luma mapping and/or chroma residual scaling, andthus video/image coding efficiency may be increased.

According to an embodiment of the present document, the LMCS proceduremay be performed even on a block having a dual tree structure, and thusthe efficiency of the LMCS may be increased. Further, coding performance(e.g., objective/subjective image quality) for a block having a dualtree structure may be improved.

According to an embodiment of the present document, granularity of (a)value(s) of a chroma residual scaling factor may be controlled throughexplicit signaling of information about the value(s) of the chromaresidual scaling factor.

According to an embodiment of the present document, chroma residualscaling may be efficiently performed through implicit derivation ofinformation about (a) value(s) of a chroma residual scaling factor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows an example of a video/image coding system towhich embodiments of the present document may be applied.

FIG. 2 is a view schematically illustrating the configuration of avideo/image encoding apparatus to which embodiments of the presentdocument may be applied.

FIG. 3 is a view schematically illustrating the configuration of avideo/image decoding apparatus to which embodiments of the presentdocument may be applied.

FIG. 4 shows an exemplary block tree architecture.

FIG. 5 exemplarily shows a hierarchical architecture for a codedvideo/image.

FIG. 6 exemplarily shows the hierarchical architecture of a CVSaccording to an embodiment of the present document.

FIG. 7 shows an exemplary LMCS architecture according to an embodimentof the present document.

FIG. 8 shows an LMCS architecture according to another embodiment of thepresent document.

FIG. 9 shows a graph showing exemplary forward mapping.

FIG. 10 is a flowchart showing a method of deriving a chroma residualscaling index in accordance with an embodiment of the present document.

FIG. 11 shows linear fitting of pivot points according to an embodimentof the present document.

FIG. 12 shows an example of a linear reshaper according to an embodimentof the present document.

FIG. 13 shows an example of linear forward mapping in an embodiment ofthe present document.

FIG. 14 shows an example of inverse forward mapping in an embodiment ofthe present document.

FIG. 15 is a view illustrating an LMCS procedure according to anembodiment of the present document.

FIGS. 16 and 17 schematically show an example of a video/image encodingmethod according to (an) embodiment(s) of the present document andrelevant components.

FIGS. 18 and 19 schematically show an example of a image/video decodingmethod according to an embodiment of the present document and relevantcomponents.

FIG. 20 shows an example of a contents streaming system to which theembodiments disclosed in the present document may be applied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present document may be modified in various forms, and specificembodiments thereof will be described and shown in the drawings.However, the embodiments are not intended for limiting the presentdocument. The terms used in the following description are used to merelydescribe specific embodiments, but are not intended to limit the presentdocument. An expression of a singular number includes an expression ofthe plural number, so long as it is clearly read differently. The termssuch as “include” and “have” are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and it should be thusunderstood that the possibility of existence or addition of one or moredifferent features, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

Meanwhile, each configuration in the drawings described in the presentdocument is shown independently for the convenience of descriptionregarding different characteristic functions, and does not mean thateach configuration is implemented as separate hardware or separatesoftware. For example, two or more components among each component maybe combined to form one component, or one component may be divided intoa plurality of components. Embodiments in which each component isintegrated and/or separated are also included in the scope of thedisclosure of the present document.

Hereinafter, examples of the present embodiment will be described indetail with reference to the accompanying drawings. In addition, likereference numerals are used to indicate like elements throughout thedrawings, and the same descriptions on the like elements will beomitted.

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

Referring to FIG. 1, a video/image coding system may include a firstdevice (a source device) and a second device (a reception device). Thesource device may transmit encoded video/image information or data tothe reception device through a digital storage medium or network in theform of a file or streaming.

The source device may include a video source, an encoding apparatus, anda transmitter. The receiving device may include a receiver, a decodingapparatus, and a renderer. The encoding apparatus may be called avideo/image encoding apparatus, and the decoding apparatus may be calleda video/image decoding apparatus. The transmitter may be included in theencoding apparatus. The receiver may be included in the decodingapparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.

The video source may acquire video/image through a process of capturing,synthesizing, or generating the video/image. The video source mayinclude a video/image capture device and/or a video/image generatingdevice. The video/image capture device may include, for example, one ormore cameras, video/image archives including previously capturedvideo/images, and the like. The video/image generating device mayinclude, for example, computers, tablets and smartphones, and may(electronically) generate video/images. For example, a virtualvideo/image may be generated through a computer or the like. In thiscase, the video/image capturing process may be replaced by a process ofgenerating related data.

The encoding apparatus may encode input video/image. The encodingapparatus may perform a series of procedures such as prediction,transform, and quantization for compaction and coding efficiency. Theencoded data (encoded video/image information) may be output in the formof a bitstream.

The transmitter may transmit the encoded image/image information or dataoutput in the form of a bitstream to the receiver of the receivingdevice through a digital storage medium or a network in the form of afile or streaming. The digital storage medium may include variousstorage mediums such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and thelike. The transmitter may include an element for generating a media filethrough a predetermined file format and may include an element fortransmission through a broadcast/communication network. The receiver mayreceive/extract the bitstream and transmit the received bitstream to thedecoding apparatus.

The decoding apparatus may decode the video/image by performing a seriesof procedures such as dequantization, inverse transform, and predictioncorresponding to the operation of the encoding apparatus.

The renderer may render the decoded video/image. The renderedvideo/image may be displayed through the display.

The present document relates to video/image coding. For example, amethod/embodiment disclosed in the present document may be applied to amethod disclosed in the versatile video coding (VVC) standard, theessential video coding (EVC) standard, the AOMedia Video 1 (AV1)standard, the 2nd generation of audio video coding standard (AVS2) orthe next generation video/image coding standard (e.g., H.267, H.268, orthe like).

The present document suggests various embodiments of video/image coding,and the above embodiments may also be performed in combination with eachother unless otherwise specified.

In the present document, a video may refer to a series of images overtime. A picture generally refers to the unit representing one image at aparticular time frame, and a slice/tile refers to the unit constitutinga part of the picture in terms of coding. A slice/tile may include oneor more coding tree units (CTUs). One picture may consist of one or moreslices/tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles. A brick may represent arectangular region of CTU rows within a tile in a picture. A tile may bepartitioned into a multiple bricks, each of which may be constructedwith one or more CTU rows within the tile. A tile that is notpartitioned into multiple bricks may also be referred to as a brick. Abrick scan may represent a specific sequential ordering of CTUspartitioning a picture, wherein the CTUs may be ordered in a CTU rasterscan within a brick, and bricks within a tile may be orderedconsecutively in a raster scan of the bricks of the tile, and tiles in apicture may be ordered consecutively in a raster scan of the tiles ofthe picture. A tile is a rectangular region of CTUs within a particulartile column and a particular tile row in a picture. The tile column is arectangular region of CTUs having a height equal to the height of thepicture and a width specified by syntax elements in the pictureparameter set. The tile row is a rectangular region of CTUs having aheight specified by syntax elements in the picture parameter set and awidth equal to the width of the picture. A tile scan is a specificsequential ordering of CTUs partitioning a picture in which the CTUs areordered consecutively in CTU raster scan in a tile whereas tiles in apicture are ordered consecutively in a raster scan of the tiles of thepicture. A slice includes an integer number of bricks of a picture thatmay be exclusively contained in a single NAL unit. A slice may consistsof either a number of complete tiles or only a consecutive sequence ofcomplete bricks of one tile. In the present document, a tile group and aslice may be used in place of each other. For example, in the presentdocument, a tile group/tile group header may be referred to as aslice/slice header.

Meanwhile, one picture may be divided into two or more subpictures. Asubpicture may be a rectangular region of one or more slices within apicture.

A pixel or a pel may mean a smallest unit constituting one picture (orimage). Also, ‘sample’ may be used as a term corresponding to a pixel. Asample may generally represent a pixel or a value of a pixel, and mayrepresent only a pixel/pixel value of a luma component or only apixel/pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit mayinclude at least one of a specific region of the picture and informationrelated to the region. One unit may include one luma block and twochroma (ex. cb, cr) blocks. The unit may be used interchangeably withterms such as block or area in some cases. In a general case, an M×Nblock may include samples (or sample arrays) or a set (or array) oftransform coefficients of M columns and N rows. Alternatively, thesample may mean a pixel value in the spatial domain, and when such apixel value is transformed to the frequency domain, it may mean atransform coefficient in the frequency domain.

In the present document, “A or B” may mean “only A”, “only B” or “both Aand B”. In other words, “A or B” in the present document may beinterpreted as “A and/or B”. For example, in the present document “A, Bor C (A, B or C)” means “only A”, “only B”, “only C”, or “anycombination of A, B and C”.

A slash (/) or comma (comma) used in the present document may mean“and/or”. For example, “A/B” may mean “A and/or B”. Accordingly, “A/B”may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C”may mean “A, B, or C”.

In the present document, “at least one of A and B” may mean “only A”,“only B” or “both A and B”. Also, in the present document, theexpression “at least one of A or B” or “at least one of A and/or B” maybe interpreted the same as “at least one of A and B”.

Also, in the present document, “at least one of A, B and C” means “onlyA”, “only B”, “only C”, or “any combination of A, B and C”. Also, “atleast one of A, B or C” or “at least one of A, B and/or C” may mean “atleast one of A, B and C”.

Also, parentheses used in the present document may mean “for example”.

Specifically, when “prediction (intra prediction)” is indicated, “intraprediction” may be proposed as an example of “prediction”. In otherwords, “prediction” in the present document is not limited to “intraprediction”, and “intra prediction” may be proposed as an example of“prediction”. Also, even when “prediction (i.e., intra prediction)” isindicated, “intra prediction” may be proposed as an example of“prediction”.

Technical features that are individually described in one drawing in thepresent document may be implemented individually or simultaneously.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied. Hereinafter, what is referred to as the videoencoding apparatus may include an image encoding apparatus.

Referring to FIG. 2, the encoding apparatus 200 includes an imagepartitioner 210, a predictor 220, a residual processor 230, and anentropy encoder 240, an adder 250, a filter 260, and a memory 270. Thepredictor 220 may include an inter predictor 221 and an intra predictor222. The residual processor 230 may include a transformer 232, aquantizer 233, a dequantizer 234, and an inverse transformer 235. Theresidual processor 230 may further include a subtractor 231. The adder250 may be called a reconstructor or a reconstructed block generator.The image partitioner 210, the predictor 220, the residual processor230, the entropy encoder 240, the adder 250, and the filter 260 may beconfigured by at least one hardware component (ex. An encoder chipset orprocessor) according to an embodiment. In addition, the memory 270 mayinclude a decoded picture buffer (DPB) or may be configured by a digitalstorage medium. The hardware component may further include the memory270 as an internal/external component.

The image partitioner 210 may partition an input image (or a picture ora frame) input to the encoding apparatus 200 into one or moreprocessors. For example, the processor may be called a coding unit (CU).In this case, the coding unit may be recursively partitioned accordingto a quad-tree binary-tree ternary-tree (QTBTTT) structure from a codingtree unit (CTU) or a largest coding unit (LCU). For example, one codingunit may be partitioned into a plurality of coding units of a deeperdepth based on a quad tree structure, a binary tree structure, and/or aternary structure. In this case, for example, the quad tree structuremay be applied first and the binary tree structure and/or ternarystructure may be applied later. Alternatively, the binary tree structuremay be applied first. The coding procedure according to the presentdisclosure may be performed based on the final coding unit that is nolonger partitioned. In this case, the largest coding unit may be used asthe final coding unit based on coding efficiency according to imagecharacteristics, or if necessary, the coding unit may be recursivelypartitioned into coding units of deeper depth and a coding unit havingan optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure of prediction, transform, andreconstruction, which will be described later. As another example, theprocessor may further include a prediction unit (PU) or a transform unit(TU). In this case, the prediction unit and the transform unit may besplit or partitioned from the aforementioned final coding unit. Theprediction unit may be a unit of sample prediction, and the transformunit may be a unit for deriving a transform coefficient and/or a unitfor deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area insome cases. In a general case, an M×N block may represent a set ofsamples or transform coefficients composed of M columns and N rows. Asample may generally represent a pixel or a value of a pixel, mayrepresent only a pixel/pixel value of a luma component or represent onlya pixel/pixel value of a chroma component. A sample may be used as aterm corresponding to one picture (or image) for a pixel or a pel.

In the encoding apparatus 200, a prediction signal (predicted block,prediction sample array) output from the inter predictor 221 or theintra predictor 222 is subtracted from an input image signal (originalblock, original sample array) to generate a residual signal residualblock, residual sample array), and the generated residual signal istransmitted to the transformer 232. In this case, as shown, a unit forsubtracting a prediction signal (predicted block, prediction samplearray) from the input image signal (original block, original samplearray) in the encoder 200 may be called a subtractor 231. The predictormay perform prediction on a block to be processed (hereinafter, referredto as a current block) and generate a predicted block includingprediction samples for the current block. The predictor may determinewhether intra prediction or inter prediction is applied on a currentblock or CU basis. As described later in the description of eachprediction mode, the predictor may generate various information relatedto prediction, such as prediction mode information, and transmit thegenerated information to the entropy encoder 240. The information on theprediction may be encoded in the entropy encoder 240 and output in theform of a bitstream.

The intra predictor 222 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The non-directional mode may include, for example,a DC mode and a planar mode. The directional mode may include, forexample, 33 directional prediction modes or 65 directional predictionmodes according to the degree of detail of the prediction direction.However, this is merely an example, more or less directional predictionmodes may be used depending on a setting. The intra predictor 222 maydetermine the prediction mode applied to the current block by using aprediction mode applied to a neighboring block.

The inter predictor 221 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. Here, in order to reduce theamount of motion information transmitted in the inter prediction mode,the motion information may be predicted in units of blocks, sub-blocks,or samples based on correlation of motion information between theneighboring block and the current block. The motion information mayinclude a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0prediction, L1 prediction, Bi prediction, etc.) information. In the caseof inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. The referencepicture including the reference block and the reference pictureincluding the temporal neighboring block may be the same or different.The temporal neighboring block may be called a collocated referenceblock, a co-located CU (colCU), and the like, and the reference pictureincluding the temporal neighboring block may be called a collocatedpicture (colPic). For example, the inter predictor 221 may configure amotion information candidate list based on neighboring blocks andgenerate information indicating which candidate is used to derive amotion vector and/or a reference picture index of the current block.Inter prediction may be performed based on various prediction modes. Forexample, in the case of a skip mode and a merge mode, the interpredictor 221 may use motion information of the neighboring block asmotion information of the current block. In the skip mode, unlike themerge mode, the residual signal may not be transmitted. In the case ofthe motion vector prediction (MVP) mode, the motion vector of theneighboring block may be used as a motion vector predictor and themotion vector of the current block may be indicated by signaling amotion vector difference.

The predictor 220 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply both intra prediction and inter prediction.This may be called combined inter and intra prediction (CIIP). Inaddition, the predictor may be based on an intra block copy (IBC)prediction mode or a palette mode for prediction of a block. The IBCprediction mode or palette mode may be used for content image/videocoding of a game or the like, for example, screen content coding (SCC).The IBC basically performs prediction in the current picture but may beperformed similarly to inter prediction in that a reference block isderived in the current picture. That is, the IBC may use at least one ofthe inter prediction techniques described in the present disclosure. Thepalette mode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The prediction signal generated by the predictor (including the interpredictor 221 and/or the intra predictor 222) may be used to generate areconstructed signal or to generate a residual signal. The transformer232 may generate transform coefficients by applying a transformtechnique to the residual signal. For example, the transform techniquemay include at least one of a discrete cosine transform (DCT), adiscrete sine transform (DST), a karhunen-loave transform (KLT), agraph-based transform (GBT), or a conditionally non-linear transform(CNT). Here, the GBT means transform obtained from a graph whenrelationship information between pixels is represented by the graph. TheCNT refers to transform generated based on a prediction signal generatedusing all previously reconstructed pixels. In addition, the transformprocess may be applied to square pixel blocks having the same size ormay be applied to blocks having a variable size rather than square.

The quantizer 233 may quantize the transform coefficients and transmitthem to the entropy encoder 240 and the entropy encoder 240 may encodethe quantized signal (information on the quantized transformcoefficients) and output a bitstream. The information on the quantizedtransform coefficients may be referred to as residual information. Thequantizer 233 may rearrange block type quantized transform coefficientsinto a one-dimensional vector form based on a coefficient scanning orderand generate information on the quantized transform coefficients basedon the quantized transform coefficients in the one-dimensional vectorform. Information on transform coefficients may be generated. Theentropy encoder 240 may perform various encoding methods such as, forexample, exponential Golomb, context-adaptive variable length coding(CAVLC), context-adaptive binary arithmetic coding (CABAC), and thelike. The entropy encoder 240 may encode information necessary forvideo/image reconstruction other than quantized transform coefficients(ex. values of syntax elements, etc.) together or separately. Encodedinformation (ex. encoded video/image information) may be transmitted orstored in units of NALs (network abstraction layer) in the form of abitstream. The video/image information may further include informationon various parameter sets such as an adaptation parameter set (APS), apicture parameter set (PPS), a sequence parameter set (SPS), or a videoparameter set (VPS). In addition, the video/image information mayfurther include general constraint information. In the presentdisclosure, information and/or syntax elements transmitted/signaled fromthe encoding apparatus to the decoding apparatus may be included invideo/picture information. The video/image information may be encodedthrough the above-described encoding procedure and included in thebitstream. The bitstream may be transmitted over a network or may bestored in a digital storage medium. The network may include abroadcasting network and/or a communication network, and the digitalstorage medium may include various storage media such as USB, SD, CD,DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown)transmitting a signal output from the entropy encoder 240 and/or astorage unit (not shown) storing the signal may be included asinternal/external element of the encoding apparatus 200, andalternatively, the transmitter may be included in the entropy encoder240.

The quantized transform coefficients output from the quantizer 233 maybe used to generate a prediction signal. For example, the residualsignal (residual block or residual samples) may be reconstructed byapplying dequantization and inverse transform to the quantized transformcoefficients through the dequantizer 234 and the inverse transformer235. The adder 250 adds the reconstructed residual signal to theprediction signal output from the inter predictor 221 or the intrapredictor 222 to generate a reconstructed signal (reconstructed picture,reconstructed block, reconstructed sample array). If there is noresidual for the block to be processed, such as a case where the skipmode is applied, the predicted block may be used as the reconstructedblock. The adder 250 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used forintra prediction of a next block to be processed in the current pictureand may be used for inter prediction of a next picture through filteringas described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied duringpicture encoding and/or reconstruction.

The filter 260 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter260 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 270, specifically, a DPB of thememory 270. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like. The filter 260 may generate variousinformation related to the filtering and transmit the generatedinformation to the entropy encoder 240 as described later in thedescription of each filtering method. The information related to thefiltering may be encoded by the entropy encoder 240 and output in theform of a bitstream.

The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 221. When the interprediction is applied through the encoding apparatus, predictionmismatch between the encoding apparatus 200 and the decoding apparatus300 may be avoided and encoding efficiency may be improved.

The DPB of the memory 270 DPB may store the modified reconstructedpicture for use as a reference picture in the inter predictor 221. Thememory 270 may store the motion information of the block from which themotion information in the current picture is derived (or encoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 221 and used as the motion information of thespatial neighboring block or the motion information of the temporalneighboring block. The memory 270 may store reconstructed samples ofreconstructed blocks in the current picture and may transfer thereconstructed samples to the intra predictor 222.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

Referring to FIG. 3, the decoding apparatus 300 may include an entropydecoder 310, a residual processor 320, a predictor 330, an adder 340, afilter 350, a memory 360. The predictor 330 may include an interpredictor 331 and an intra predictor 332. The residual processor 320 mayinclude a dequantizer 321 and an inverse transformer 321. The entropydecoder 310, the residual processor 320, the predictor 330, the adder340, and the filter 350 may be configured by a hardware component (ex. Adecoder chipset or a processor) according to an embodiment. In addition,the memory 360 may include a decoded picture buffer (DPB) or may beconfigured by a digital storage medium. The hardware component mayfurther include the memory 360 as an internal/external component.

When a bitstream including video/image information is input, thedecoding apparatus 300 may reconstruct an image corresponding to aprocess in which the video/image information is processed in theencoding apparatus of FIG. 2. For example, the decoding apparatus 300may derive units/blocks based on block partition related informationobtained from the bitstream. The decoding apparatus 300 may performdecoding using a processor applied in the encoding apparatus. Thus, theprocessor of decoding may be a coding unit, for example, and the codingunit may be partitioned according to a quad tree structure, binary treestructure and/or ternary tree structure from the coding tree unit or thelargest coding unit. One or more transform units may be derived from thecoding unit. The reconstructed image signal decoded and output throughthe decoding apparatus 300 may be reproduced through a reproducingapparatus.

The decoding apparatus 300 may receive a signal output from the encodingapparatus of FIG. 2 in the form of a bitstream, and the received signalmay be decoded through the entropy decoder 310. For example, the entropydecoder 310 may parse the bitstream to derive information (ex.video/image information) necessary for image reconstruction (or picturereconstruction). The video/image information may further includeinformation on various parameter sets such as an adaptation parameterset (APS), a picture parameter set (PPS), a sequence parameter set(SPS), or a video parameter set (VPS). In addition, the video/imageinformation may further include general constraint information. Thedecoding apparatus may further decode picture based on the informationon the parameter set and/or the general constraint information.Signaled/received information and/or syntax elements described later inthe present disclosure may be decoded may decode the decoding procedureand obtained from the bitstream. For example, the entropy decoder 310decodes the information in the bitstream based on a coding method suchas exponential Golomb coding, CAVLC, or CABAC, and output syntaxelements required for image reconstruction and quantized values oftransform coefficients for residual. More specifically, the CABACentropy decoding method may receive a bin corresponding to each syntaxelement in the bitstream, determine a context model using a decodingtarget syntax element information, decoding information of a decodingtarget block or information of a symbol/bin decoded in a previous stage,and perform an arithmetic decoding on the bin by predicting aprobability of occurrence of a bin according to the determined contextmodel, and generate a symbol corresponding to the value of each syntaxelement. In this case, the CABAC entropy decoding method may update thecontext model by using the information of the decoded symbol/bin for acontext model of a next symbol/bin after determining the context model.The information related to the prediction among the information decodedby the entropy decoder 310 may be provided to the predictor (the interpredictor 332 and the intra predictor 331), and the residual value onwhich the entropy decoding was performed in the entropy decoder 310,that is, the quantized transform coefficients and related parameterinformation, may be input to the residual processor 320. The residualprocessor 320 may derive the residual signal (the residual block, theresidual samples, the residual sample array). In addition, informationon filtering among information decoded by the entropy decoder 310 may beprovided to the filter 350. Meanwhile, a receiver (not shown) forreceiving a signal output from the encoding apparatus may be furtherconfigured as an internal/external element of the decoding apparatus300, or the receiver may be a component of the entropy decoder 310.Meanwhile, the decoding apparatus according to the present disclosuremay be referred to as a video/image/picture decoding apparatus, and thedecoding apparatus may be classified into an information decoder(video/image/picture information decoder) and a sample decoder(video/image/picture sample decoder). The information decoder mayinclude the entropy decoder 310, and the sample decoder may include atleast one of the dequantizer 321, the inverse transformer 322, the adder340, the filter 350, the memory 360, the inter predictor 332, and theintra predictor 331.

The dequantizer 321 may dequantize the quantized transform coefficientsand output the transform coefficients. The dequantizer 321 may rearrangethe quantized transform coefficients in the form of a two-dimensionalblock form. In this case, the rearrangement may be performed based onthe coefficient scanning order performed in the encoding apparatus. Thedequantizer 321 may perform dequantization on the quantized transformcoefficients by using a quantization parameter (ex. quantization stepsize information) and obtain transform coefficients.

The inverse transformer 322 inversely transforms the transformcoefficients to obtain a residual signal (residual block, residualsample array).

The predictor may perform prediction on the current block and generate apredicted block including prediction samples for the current block. Thepredictor may determine whether intra prediction or inter prediction isapplied to the current block based on the information on the predictionoutput from the entropy decoder 310 and may determine a specificintra/inter prediction mode.

The predictor 320 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply intra prediction and inter prediction. Thismay be called combined inter and intra prediction (CIIP). In addition,the predictor may be based on an intra block copy (IBC) prediction modeor a palette mode for prediction of a block. The IBC prediction mode orpalette mode may be used for content image/video coding of a game or thelike, for example, screen content coding (SCC). The IBC basicallyperforms prediction in the current picture but may be performedsimilarly to inter prediction in that a reference block is derived inthe current picture. That is, the IBC may use at least one of the interprediction techniques described in the present disclosure. The palettemode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The intra predictor 331 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The intra predictor 331 may determine theprediction mode applied to the current block by using a prediction modeapplied to a neighboring block.

The inter predictor 332 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. In this case, in order to reducethe amount of motion information transmitted in the inter predictionmode, motion information may be predicted in units of blocks,sub-blocks, or samples based on correlation of motion informationbetween the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index.The motion information may further include inter prediction direction(L0 prediction, L1 prediction, Bi prediction, etc.) information. In thecase of inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. For example, theinter predictor 332 may configure a motion information candidate listbased on neighboring blocks and derive a motion vector of the currentblock and/or a reference picture index based on the received candidateselection information. Inter prediction may be performed based onvarious prediction modes, and the information on the prediction mayinclude information indicating a mode of inter prediction for thecurrent block.

The adder 340 may generate a reconstructed signal (reconstructedpicture, reconstructed block, reconstructed sample array) by adding theobtained residual signal to the prediction signal (predicted block,predicted sample array) output from the predictor (including the interpredictor 332 and/or the intra predictor 331). If there is no residualfor the block to be processed, such as when the skip mode is applied,the predicted block may be used as the reconstructed block.

The adder 340 may be called reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for intraprediction of a next block to be processed in the current picture, maybe output through filtering as described below, or may be used for interprediction of a next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in thepicture decoding process.

The filter 350 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter350 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 360, specifically, a DPB of thememory 360. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 360may be used as a reference picture in the inter predictor 332. Thememory 360 may store the motion information of the block from which themotion information in the current picture is derived (or decoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 260 so as to be utilized as the motion informationof the spatial neighboring block or the motion information of thetemporal neighboring block. The memory 360 may store reconstructedsamples of reconstructed blocks in the current picture and transfer thereconstructed samples to the intra predictor 331.

In the present document, the embodiments described in the filter 260,the inter predictor 221, and the intra predictor 222 of the encodingapparatus 200 may be the same as or respectively applied to correspondto the filter 350, the inter predictor 332, and the intra predictor 331of the decoding apparatus 300. The same may also apply to the unit 332and the intra predictor 331.

As described above, in video coding, prediction is performed to increasecompression efficiency. Through this, it is possible to generate apredicted block including prediction samples for a current block, whichis a block to be coded. Here, the predicted block includes predictionsamples in a spatial domain (or pixel domain). The predicted block isderived equally from the encoding device and the decoding device, andthe encoding device decodes information (residual information) on theresidual between the original block and the predicted block, not theoriginal sample value of the original block itself. By signaling to thedevice, image coding efficiency can be increased. The decoding apparatusmay derive a residual block including residual samples based on theresidual information, and generate a reconstructed block includingreconstructed samples by summing the residual block and the predictedblock, and generate a reconstructed picture including reconstructedblocks.

The residual information may be generated through transformation andquantization processes. For example, the encoding apparatus may derive aresidual block between the original block and the predicted block, andperform a transform process on residual samples (residual sample array)included in the residual block to derive transform coefficients, andthen, by performing a quantization process on the transformcoefficients, derive quantized transform coefficients to signal theresidual related information to the decoding apparatus (via abitstream). Here, the residual information may include locationinformation, a transform technique, a transform kernel, and aquantization parameter, value information of the quantized transformcoefficients etc. The decoding apparatus may performdequantization/inverse transformation process based on the residualinformation and derive residual samples (or residual blocks). Thedecoding apparatus may generate a reconstructed picture based on thepredicted block and the residual block. The encoding apparatus may alsodequantize/inverse transform the quantized transform coefficients forreference for inter prediction of a later picture to derive a residualblock, and generate a reconstructed picture based thereon.

In the present document, at least one of quantization/dequantizationand/or transform/inverse transform may be omitted. When thequantization/dequantization is omitted, the quantized transformcoefficient may be referred to as a transform coefficient. When thetransform/inverse transform is omitted, the transform coefficients maybe called coefficients or residual coefficients, or may still be calledtransform coefficients for uniformity of expression.

In the present document, a quantized transform coefficient and atransform coefficient may be referred to as a transform coefficient anda scaled transform coefficient, respectively. In this case, the residualinformation may include information on transform coefficient(s), and theinformation on the transform coefficient(s) may be signaled throughresidual coding syntax. Transform coefficients may be derived based onthe residual information (or information on the transformcoefficient(s)), and scaled transform coefficients may be derivedthrough inverse transform (scaling) on the transform coefficients.Residual samples may be derived based on an inverse transform(transform) of the scaled transform coefficients. This may beapplied/expressed in other parts of the present document as well.

Intra prediction may refer to prediction that generates predictionsamples for the current block based on reference samples in a picture towhich the current block belongs (hereinafter, referred to as a currentpicture). When intra prediction is applied to the current block,neighboring reference samples to be used for intra prediction of thecurrent block may be derived. The neighboring reference samples of thecurrent block may include samples adjacent to the left boundary of thecurrent block having a size of nW×nH and a total of 2×nH samplesneighboring the bottom-left, samples adjacent to the top boundary of thecurrent block and a total of 2×nW samples neighboring the top-right, andone sample neighboring the top-left of the current block. Alternatively,the neighboring reference samples of the current block may include aplurality of upper neighboring samples and a plurality of leftneighboring samples. In addition, the neighboring reference samples ofthe current block may include a total of nH samples adjacent to theright boundary of the current block having a size of nW×nH, a total ofnW samples adjacent to the bottom boundary of the current block, and onesample neighboring (bottom-right) neighboring bottom-right of thecurrent block.

However, some of the neighboring reference samples of the current blockmay not be decoded yet or available. In this case, the decoder mayconfigure the neighboring reference samples to use for prediction bysubstituting the samples that are not available with the availablesamples. Alternatively, neighboring reference samples to be used forprediction may be configured through interpolation of the availablesamples.

When the neighboring reference samples are derived, (i) the predictionsample may be derived based on the average or interpolation ofneighboring reference samples of the current block, and (ii) theprediction sample may be derived based on the reference sample presentin a specific (prediction) direction for the prediction sample among theperiphery reference samples of the current block. The case of (i) may becalled non-directional mode or non-angular mode and the case of (ii) maybe called directional mode or angular mode.

Furthermore, the prediction sample may also be generated throughinterpolation between the second neighboring sample and the firstneighboring sample located in a direction opposite to the predictiondirection of the intra prediction mode of the current block based on theprediction sample of the current block among the neighboring referencesamples. The above case may be referred to as linear interpolation intraprediction (LIP). In addition, chroma prediction samples may begenerated based on luma samples using a linear model. This case may becalled LM mode.

In addition, a temporary prediction sample of the current block may bederived based on filtered neighboring reference samples, and at leastone reference sample derived according to the intra prediction modeamong the existing neighboring reference samples, that is, unfilteredneighboring reference samples, and the temporary prediction sample maybe weighted-summed to derive the prediction sample of the current block.The above case may be referred to as position dependent intra prediction(PDPC).

In addition, a reference sample line having the highest predictionaccuracy among the neighboring multi-reference sample lines of thecurrent block may be selected to derive the prediction sample by usingthe reference sample located in the prediction direction on thecorresponding line, and then the reference sample line used herein maybe indicated (signaled) to the decoding apparatus, thereby performingintra-prediction encoding. The above case may be referred to asmulti-reference line (MRL) intra prediction or MRL based intraprediction.

In addition, intra prediction may be performed based on the same intraprediction mode by dividing the current block into vertical orhorizontal subpartitions, and neighboring reference samples may bederived and used in the subpartition unit. That is, in this case, theintra prediction mode for the current block is equally applied to thesubpartitions, and the intra prediction performance may be improved insome cases by deriving and using the neighboring reference samples inthe subpartition unit. Such a prediction method may be called intrasub-partitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intraprediction type separately from the intra prediction mode. The intraprediction type may be called in various terms such as an intraprediction technique or an additional intra prediction mode. Forexample, the intra prediction type (or additional intra prediction mode)may include at least one of the above-described LIP, PDPC, MRL, and ISP.A general intra prediction method except for the specific intraprediction type such as LIP, PDPC, MRL, or ISP may be called a normalintra prediction type. The normal intra prediction type may be generallyapplied when the specific intra prediction type is not applied, andprediction may be performed based on the intra prediction mode describedabove. Meanwhile, post-filtering may be performed on the predictedsample derived as needed.

Specifically, the intra prediction procedure may include an intraprediction mode/type determination step, a neighboring reference samplederivation step, and an intra prediction mode/type based predictionsample derivation step. In addition, a post-filtering step may beperformed on the predicted sample derived as needed.

When intra prediction is applied, the intra prediction mode applied tothe current block may be determined using the intra prediction mode ofthe neighboring block. For example, the decoding apparatus may selectone of most probable mode (mpm) candidates of an mpm list derived basedon the intra prediction mode of the neighboring block (ex. left and/orupper neighboring blocks) of the current block based on the received mpmindex and select one of the other remaining intro prediction modes notincluded in the mpm candidates (and planar mode) based on the remainingintra prediction mode information. The mpm list may be configured toinclude or not include a planar mode as a candidate. For example, if thempm list includes the planar mode as a candidate, the mpm list may havesix candidates. If the mpm list does not include the planar mode as acandidate, the mpm list may have three candidates. When the mpm listdoes not include the planar mode as a candidate, a not planar flag (ex.intra_luma_not_planar_flag) indicating whether an intra prediction modeof the current block is not the planar mode may be signaled. Forexample, the mpm flag may be signaled first, and the mpm index and notplanar flag may be signaled when the value of the mpm flag is 1. Inaddition, the mpm index may be signaled when the value of the not planarflag is 1. Here, the mpm list is configured not to include the planarmode as a candidate does not is to signal the not planar flag first tocheck whether it is the planar mode first because the planar mode isalways considered as mpm.

For example, whether the intra prediction mode applied to the currentblock is in mpm candidates (and planar mode) or in remaining mode may beindicated based on the mpm flag (ex. Intra_luma_mpm_flag). A value 1 ofthe mpm flag may indicate that the intra prediction mode for the currentblock is within mpm candidates (and planar mode), and a value 0 of thempm flag may indicate that the intra prediction mode for the currentblock is not in the mpm candidates (and planar mode). The value 0 of thenot planar flag (ex. Intra_luma_not_planar_flag) may indicate that theintra prediction mode for the current block is planar mode, and thevalue 1 of the not planar flag value may indicate that the intraprediction mode for the current block is not the planar mode. The mpmindex may be signaled in the form of an mpm_idx or intra_luma_mpm_idxsyntax element, and the remaining intra prediction mode information maybe signaled in the form of a rem_intra_luma_pred_mode orintra_luma_mpm_remainder syntax element. For example, the remainingintra prediction mode information may index remaining intra predictionmodes not included in the mpm candidates (and planar mode) among allintra prediction modes in order of prediction mode number to indicateone of them. The intra prediction mode may be an intra prediction modefor a luma component (sample). Hereinafter, intra prediction modeinformation may include at least one of the mpm flag (ex.Intra_luma_mpm_flag), the not planar flag (ex.Intra_luma_not_planar_flag), the mpm index (ex. mpm_idx orintra_luma_mpm_idx), and the remaining intra prediction mode information(rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the presentdocument, the MPM list may be referred to in various terms such as MPMcandidate list and candModeList. When MIP is applied to the currentblock, a separate mpm flag (ex. intra_mip_mpm_flag), an mpm index (ex.intra_mip_mpm_idx), and remaining intra prediction mode information (ex.intra_mip_mpm_remainder) for MIP may be signaled and the not planar flagis not signaled.

In other words, in general, when block splitting is performed on animage, a current block and a neighboring block to be coded have similarimage characteristics. Therefore, the current block and the neighboringblock have a high probability of having the same or similar intraprediction mode. Thus, the encoder may use the intra prediction mode ofthe neighboring block to encode the intra prediction mode of the currentblock.

For example, the encoder/decoder may configure a list of most probablemodes (MPM) for the current block. The MPM list may also be referred toas an MPM candidate list. Herein, the MPM may refer to a mode used toimprove coding efficiency in consideration of similarity between thecurrent block and neighboring block in intra prediction mode coding. Asdescribed above, the MPM list may be configured to include the planarmode or may be configured to exclude the planar mode. For example, whenthe MPM list includes the planar mode, the number of candidates in theMPM list may be 6. And, if the MPM list does not include the planarmode, the number of candidates in the MPM list may be 5.

The encoder/decoder may configure an MPM list including 5 or 6 MPMs.

In order to configure the MPM list, three types of modes can beconsidered: default intra modes, neighbor intra modes, and the derivedintra modes.

For the neighboring intra modes, two neighboring blocks, i.e., a leftneighboring block and an upper neighboring block, may be considered.

As described above, if the MPM list is configured not to include theplanar mode, the planar mode is excluded from the list, and the numberof MPM list candidates may be set to 5.

In addition, the non-directional mode (or non-angular mode) among theintra prediction modes may include a DC mode based on the average ofneighboring reference samples of the current block or a planar modebased on interpolation.

When inter prediction is applied, the predictor of the encodingapparatus/decoding apparatus may derive a prediction sample byperforming inter prediction in units of blocks. Inter prediction may bea prediction derived in a manner that is dependent on data elements (ex.sample values or motion information) of picture(s) other than thecurrent picture. When inter prediction is applied to the current block,a predicted block (prediction sample array) for the current block may bederived based on a reference block (reference sample array) specified bya motion vector on the reference picture indicated by the referencepicture index. Here, in order to reduce the amount of motion informationtransmitted in the inter prediction mode, the motion information of thecurrent block may be predicted in units of blocks, subblocks, or samplesbased on correlation of motion information between the neighboring blockand the current block. The motion information may include a motionvector and a reference picture index. The motion information may furtherinclude inter prediction type (L0 prediction, L1 prediction, Biprediction, etc.) information. In the case of inter prediction, theneighboring block may include a spatial neighboring block present in thecurrent picture and a temporal neighboring block present in thereference picture. The reference picture including the reference blockand the reference picture including the temporal neighboring block maybe the same or different. The temporal neighboring block may be called acollocated reference block, a co-located CU (colCU), and the like, andthe reference picture including the temporal neighboring block may becalled a collocated picture (colPic). For example, a motion informationcandidate list may be configured based on neighboring blocks of thecurrent block, and flag or index information indicating which candidateis selected (used) may be signaled to derive a motion vector and/or areference picture index of the current block. Inter prediction may beperformed based on various prediction modes. For example, in the case ofa skip mode and a merge mode, the motion information of the currentblock may be the same as motion information of the neighboring block. Inthe skip mode, unlike the merge mode, the residual signal may not betransmitted. In the case of the motion vector prediction (MVP) mode, themotion vector of the selected neighboring block may be used as a motionvector predictor and the motion vector of the current block may besignaled. In this case, the motion vector of the current block may bederived using the sum of the motion vector predictor and the motionvector difference.

The motion information may include L0 motion information and/or L1motion information according to an inter prediction type (L0 prediction,L1 prediction, Bi prediction, etc.). The motion vector in the L0direction may be referred to as an L0 motion vector or MVL0, and themotion vector in the L1 direction may be referred to as an L1 motionvector or MVL1. Prediction based on the L0 motion vector may be calledL0 prediction, prediction based on the L1 motion vector may be called L1prediction, and prediction based on both the L0 motion vector and the L1motion vector may be called bi-prediction. Here, the L0 motion vectormay indicate a motion vector associated with the reference picture listL0 (L0), and the L1 motion vector may indicate a motion vectorassociated with the reference picture list L1 (L1). The referencepicture list L0 may include pictures that are earlier in output orderthan the current picture as reference pictures, and the referencepicture list L1 may include pictures that are later in the output orderthan the current picture. The previous pictures may be called forward(reference) pictures, and the subsequent pictures may be called reverse(reference) pictures. The reference picture list L0 may further includepictures that are later in the output order than the current picture asreference pictures. In this case, the previous pictures may be indexedfirst in the reference picture list L0 and the subsequent pictures maybe indexed later. The reference picture list L1 may further includeprevious pictures in the output order than the current picture asreference pictures. In this case, the subsequent pictures may be indexedfirst in the reference picture list 1 and the previous pictures may beindexed later. The output order may correspond to picture order count(POC) order.

FIG. 4 shows an exemplary block tree structure. FIG. 4 illustrativelyshows that a CTU is divided into multiple CUs based on quad-tree andnested multi-type tree structure.

Bold block edges represent quad-tree partitioning, and the rest of theedges represent multi-type tree partitioning. Quad-tree partition withnested multi-type tree may provide a content-adaptive coding treestructure. A CU may correspond to a coding block (CB). Alternatively,the CU may include a coding block of luma samples, and two coding blocksof corresponding chroma samples. A size of the CU may be as large asthat of a CTU, or may be as small as that of 4×4 in a luma sample unit.For example, in the case of 4:2:0 colour format (or chroma format), amaximum chroma CB size may be 64×64, and a minimum chroma CB size may be2×2.

In the present document, for example, a maximum allowed luma TB size maybe 64×64, and a maximum allowed chroma TB size may be 32×32. When awidth or a height of a CB divided according to the tree structure isgreater than a maximum transform width or a maximum transform height,the corresponding CB may be automatically (or implicitly) divided untilTB size limits in horizontal and vertical directions are satisfied.

In the present document, the coding tree scheme may support that theluma and chroma (component) blocks have separate block tree structures.A block having a separate block tree structure may be one that has beencoded into a separate tree. The case where the luma and chroma blocks inone CTU have the same block tree structure may be represented asSINGLE_TREE (single tree structure). The case where the luma and chromablocks in one CTU have separate block tree structures may be representedas DUAL_TREE (dual tree structure). In this case, the block tree typefor the luma component may be called DUAL_TREE_LUMA, and the block treetype for the chroma component may be called DUAL_TREE_CHROMA. A blockhaving a dual tree structure may be one that has been coded into a dualtree. For P and B slice/tile groups, luma and chroma CTBs in one CTU maybe restricted to have the same coding tree structure. However, for Islices/tile groups, luma and chroma blocks may have separate block treestructures, respectively. When a separate block tree mode is applied, aluma CTB may be divided into CUs based on a specific coding treestructure, and a chroma CTB may be divided into chroma CUs based onanother coding tree structure. This may mean that a CU in an Islice/tile group is constituted by a coding block of a luma component orcoding blocks of two chroma components, and that a CU of a P or Bslice/tile group is constituted by blocks of three color components. Inthis document, a slice may be referred to as a tile/tile group, and atile/tile group may be referred to as a slice.

Although the quadtree coding tree structure accompanied by the multitypetree has been described, the structure in which the CU is divided is notlimited thereto. For example, the BT structure and the TT structure maybe interpreted as a concept included in multiple partitioning tree (MPT)structure, and the CU may be interpreted as being divided through the QTstructure and the MPT structure. In an example in which a CU is dividedthrough a QT structure and an MPT structure, the partitioning structuremay be determined by signaling a syntax element (e.g., MPT_split_type)including information on how many blocks a leaf node of the QT structureis divided into, and a syntax element (e.g., MPT_split_mode) includinginformation on whether a leaf node of the QT structure is divided in avertical direction or in a horizontal direction.

Further, in another example, a CU may be divided with any other methodthan the QT structure, the BT structure or the TT structure. That is,unlike that a CU at a lower depth is divided into ¼ size of a CU at ahigher depth according to the QT structure, or that a CU at a lowerdepth is divided into ½ size of a CU at a higher depth according to theBT structure, or that a CU at a lower depth is divided into ¼ or ½ sizeof a CU at a higher depth according to the TT structure, a CU at a lowerdepth may be divided into ⅕, ⅓, ⅜, ⅗, ⅔ or ⅝ size of a CU at a higherdepth according to circumstances, and the CU partitioning method is notlimited to this.

FIG. 5 exemplarily shows a hierarchical structure for a codedimage/video.

Referring to FIG. 5, coded image/video is divided into a video codinglayer (VCL) that handles the decoding process of the image/video anditself, a subsystem that transmits and stores the coded information, andNAL (network abstraction layer) in charge of function and presentbetween the VCL and the subsystem.

In the VCL, VCL data including compressed image data (slice data) isgenerated, or a parameter set including a picture parameter set (PSP), asequence parameter set (SPS), and a video parameter set (VPS) or asupplemental enhancement information (SEI) message additionally requiredfor an image decoding process may be generated.

In the NAL, a NAL unit may be generated by adding header information(NAL unit header) to a raw byte sequence payload (RBSP) generated in aVCL. In this case, the RBSP refers to slice data, parameter set, SEImessage, etc., generated in the VCL. The NAL unit header may include NALunit type information specified according to RBSP data included in thecorresponding NAL unit.

As shown in the figure, the NAL unit may be classified into a VCL NALunit and a Non-VCL NAL unit according to the RBSP generated in the VCL.The VCL NAL unit may mean a NAL unit that includes information on theimage (slice data) on the image, and the Non-VCL NAL unit may mean a NALunit that includes information (parameter set or SEI message) requiredfor decoding the image.

The above-described VCL NAL unit and Non-VCL NAL unit may be transmittedthrough a network by attaching header information according to the datastandard of the subsystem. For example, the NAL unit may be transformedinto a data format of a predetermined standard such as an H.266/VVC fileformat, a real-time transport protocol (RTP), a transport stream (TS),etc., and transmitted through various networks.

As described above, the NAL unit may be specified with the NAL unit typeaccording to the RBSP data structure included in the corresponding NALunit, and information on the NAL unit type may be stored and signaled inthe NAL unit header.

For example, the NAL unit may be classified into a VCL NAL unit type anda Non-VCL NAL unit type according to whether the NAL unit includesinformation (slice data) about an image. The VCL NAL unit type may beclassified according to the nature and type of pictures included in theVCL NAL unit, and the Non-VCL NAL unit type may be classified accordingto types of parameter sets.

The following is an example of the NAL unit type specified according tothe type of parameter set included in the Non-VCL NAL unit type.

APS (Adaptation Parameter Set) NAL unit: Type for NAL unit including APS

DPS (Decoding Parameter Set) NAL unit: Type for NAL unit including DPS

VPS (Video Parameter Set) NAL unit: Type for NAL unit including VPS

SPS(Sequence Parameter Set) NAL unit: Type for NAL unit including SPS

PPS (Picture Parameter Set) NAL unit: Type for NAL unit including PPS

PH (Picture header) NAL unit: Type for NAL unit including PH

The aforementioned NAL unit types may have syntax information for theNAL unit type, and the syntax information may be stored and signaled ina NAL unit header. For example, the syntax information may benal_unit_type, and NAL unit types may be specified by a nal_unit_typevalue.

Meanwhile, as described above, one picture may include a plurality ofslices, and one slice may include a slice header and slice data. In thiscase, one picture header may be further added to a plurality of slices(a slice header and a slice data set) in one picture. The picture header(picture header syntax) may include information/parameters commonlyapplicable to the picture. In the present document, a slice may be mixedor replaced with a tile group. Also, in the present document, a sliceheader may be mixed or replaced with a tile group header.

The slice header (slice header syntax) may includeinformation/parameters that may be commonly applied to the slice. TheAPS (APS syntax) or the PPS (PPS syntax) may includeinformation/parameters that may be commonly applied to one or moreslices or pictures. The SPS (SPS syntax) may includeinformation/parameters that may be commonly applied to one or moresequences. The VPS (VPS syntax) may include information/parameters thatmay be commonly applied to multiple layers. The DPS (DPS syntax) mayinclude information/parameters that may be commonly applied to theoverall video. The DPS may include information/parameters related toconcatenation of a coded video sequence (CVS). The high level syntax(HLS) in the present document may include at least one of the APSsyntax, the PPS syntax, the SPS syntax, the VPS syntax, the DPS syntax,and the slice header syntax.

In the present document, the image/image information encoded from theencoding apparatus and signaled to the decoding apparatus in the form ofa bitstream includes not only partitioning related information in apicture, intra/inter prediction information, residual information,in-loop filtering information, etc, but also information included in aslice header, information included in the APS, information included inthe PPS, information included in an SPS, and/or information included inthe VPS.

Meanwhile, in order to compensate for a difference between an originalimage and a reconstructed image due to an error occurring in acompression coding process such as quantization, an in-loop filteringprocess may be performed on reconstructed samples or reconstructedpictures as described above. As described above, the in-loop filteringmay be performed by the filter of the encoding apparatus and the filterof the decoding apparatus, and a deblocking filter, SAO, and/or adaptiveloop filter (ALF) may be applied. For example, the ALF process may beperformed after the deblocking filtering process and/or the SAO processare completed. However, even in this case, the deblocking filteringprocess and/or the SAO process may be omitted.

Meanwhile, in order to increase coding efficiency, luma mapping wthchroma scaling (LMCS) may be applied as described above. LMCS may bereferred to as a loop reshaper (reshaping). In order to increase codingefficiency, LMCS control and/or signaling of LMCS related informationmay be performed hierarchically.

FIG. 6 exemplarily shows a hierarchical structure of CVS according to anembodiment of this document. A coded video sequence (CVS) may include anSPS, a PPS, a tile group header, tile data, and/or CTU(s). Here, thetile group header and the tile data may be referred to as a slice headerand slice data, respectively.

The SPS may include flags natively to enable tools to be used in CVS. Inaddition, the SPS may be referred to by the PPS including information onparameters that change for each picture. Each of the coded pictures mayinclude one or more coded rectangular domain tiles. The tiles may begrouped into raster scans forming tile groups. Each tile group isencapsulated with header information called a tile group header. Eachtile consists of a CTU comprising coded data. Here the data may includeoriginal sample values, prediction sample values, and its luma andchroma components (luma prediction sample values and chroma predictionsample values).

According to the existing method, ALF data (ALF parameter) or LMCS data(LMCS parameter) was incorporated in the tile group header. Because onevideo is constituted by multiple pictures and one picture includesmultiple tiles, signaling ALF data (ALF parameter) or LMCS data (LMCSparameter) frequently in units of tile groups has led to the problem ofthe reduction of the coding efficiency.

According to an embodiment proposed in this document, the ALF parameteror LMCS data (LMCS parameter) may be incorporated in an APS andsignaled.

In an embodiment, an APS is defined and the APS may carry necessary ALFdata (ALF parameters). In addition, the APS may have aself-identification parameter and ALF data. The self-identificationparameter of the APS may include an APS ID. That is, the APS may includeinformation showing the APS ID other than an ALF data field. A tilegroup header or a slice header may refer to the APS using APS indexinformation. In other words, the tile group header or the slice headermay include APS index information and may perform an ALF procedure for atarget block on the basis of the ALF data (ALF parameters) included inan APS having an APS ID indicated by the APS index information. In thiscase, the APS index information may be referred to as APS IDinformation.

Further, the SPS may include a flag allowing for use of an ALF. Forexample, when CVS begins, an SPS may be checked and the flag in the SPSmay be checked. For example, the SPS may include the syntaxes of thefollowing Table 1. The syntaxes of Table 1 may be a portion of the SPS.

TABLE 1 seq_parameter_set_rbsp( ) { Descriptor ...  sps_alf_enabled_flagu(1) }

The semantics of syntax elements included in the syntax of Table 1 maybe represented, for example, as shown in the following table.

TABLE 2 sps_alf_enabled_flag equal to 0 specifies that the adaptive loopfilter is disabled sps_alf_enabled_flag equal to 1 specifies that theadaptive loop filter is enabled

That is, the sps_alf_enabled_flag syntax element may indicate whether ornot the ALF is enabled, based on whether the value thereof is 0 or 1.The sps_alf_enabled_flag syntax element may be referred to as an ALFenabled flag (which may be referred to as a first ALF enabled flag), andmay be included in the SPS. That is, the ALF enabled flag may besignaled in the SPS (or SPS level). When the value of the ALF enabledflag signaled in the SPS is 1, the ALF may be determined to be basicallyenabled for pictures in the CVS making reference to the SPS. Meanwhile,as described above, the ALF may be separately turned on/off by signalingan additional enabled flag at a lower level than the SPS.

For example, if the ALF tool is enabled for CVS, an additional enabledflag (which may be called a second ALF enabled flag) may be signaled ina tile group header or a slice header. The second ALF enabled flag maybe parsed/signaled when, for example, the ALF is enabled at the SPSlevel. When the value of the second ALF enabled flag is 1, ALF data maybe parsed through the tile group header or the slice header. Forexample, the second ALF enabled flag may specify an ALF enabledcondition for luma and chroma components. The ALF data may be accessedthrough APS ID information.

TABLE 3 tile_group_header( ) { Descriptor tile_group_pic_parameter_set_id ue(v)  if( sps_alf_enabled_flag ) {  tile_group_alf_enabled_flag u(1)   if( tile_group_alf_enabled_flag )   tile_group_aps_id u(5)  }

TABLE 4 slice_header( ) { Descriptor  slice_pic_parameter_set_id ue(v) if( sps_alf_enabled_flag ) {   slice_alf_enabled_flag u(1)   if(slice_alf_enabled_flag )    slice_aps_id u(5)  }

The semantics of syntax elements included in the syntax of Table 3 or 4may be represented, for example, as shown in the following tables.

TABLE 5 tile_group_alf_enabled flag equal to 1 specifies that adaptiveloop filter is enabled and may be applied to Y, Cb, or Cr in a tilegroup. tile_group_aps_id specifies that adaptive parameter set id of APSthat the tile group refers to.The TemporalId of the APS NAL unit havingadaptation parameter set id equal to tile group aps id shall be lessthan or equal to the TemporalId of the coded tile group NAL unit. Whenmultiple APSs with the same value of adaptation parameter set id arereferred to by two or more tile groups of the same picture, the multipleAPSs with the same value of adaptation parameter set id shall have thesame content.

TABLE 6 slice_alf_enabled_flag equal to 1 specifies that adaptive loopfilter is enabled and may be applied to Y, Cb, or Cr color component ina slice. slice_alf_enabled_flag equal to 0 specifies that adaptive loopfilter is disabled for all color components in a slice. slice_aps_idspecifies the adaptation_parameter_set_id of the APS that the slicerefers to. The TemporalId of the APS NAL unit havingadaptation_parameter_set_id equal to tile group aps id shall be lessthan or equal to the TemporalID of the coded slice NAL unit. Whenmultiple APSs with the same value of adaptation parameter set id arereferred to by two or more slices of the same picture, the multiple APSswith the same value of adaptation parameter set id shall have the samecontent.

The second ALF enabled flag may include a tile_group_alf_enabled_flagsyntax element or a slice_alf_enabled_flag syntax element.

An APS referenced by a corresponding tile group or a corresponding slicemay be identified based on the APS ID information (e.g., atile_group_aps_id syntax element or a slice_aps_id syntax element). TheAPS may include ALF data.

Meanwhile, the structure of the APS including the ALF data may bedescribed based on the following syntax and semantics, for example. Thesyntax of Table 7 may be a part of the APS.

TABLE 7 adaptation_parameter_set_rbsp( ) { Descriptor adaptation_parameter_set_id u(5)  alf_data( )  aps_extension_flag u(1) if( aps_extension_flag )   while( more_rbsp_data( ) )   aps_extension_data_flag u(1)  rbsp_trailing_bits( ) }

TABLE 8 adaptation_parameter_set_id provides an identifier for the APSfor reference by other syntax elements.  NOTE - APSs can be sharedacross pictures and can be different in  different tile groups within apicture. aps_extension_flag equal to 0 specifies that noaps_extension_data_flag syntax elements are present in the APS RBSPsyntax structure. aps_ extension_flag equal to 1 specifies that thereare aps_extension_data_flag syntax elements present in the APS RBSPsyntax structure. aps_extension_data_flag may have any value. Itspresence and value do not affect decoder conformance to profilesspecified in this version of this Specification. Decoders conforming tothis version of this Specification shall ignore allaps_extension_data_flag syntax elements.

As described above, the adaptation_parameter_set_id syntax element mayindicate the identifier of the corresponding APS. That is, the APS maybe identified based on the adaptation_parameter_set_id syntax element.The adaptation_parameter_set_id syntax element may be referred to as APSID information. Also, the APS may include an ALF data field. The ALFdata field may be parsed/signaled after the adaptation_parameter_set_idsyntax element.

Additionally, for example, an APS extension flag (ex. aps_extension_flagsyntax element) may be parsed/signaled in the APS. The APS extensionflag may indicate whether or not APS extension data flag(aps_extension_data_flag) syntax elements are present. The APS extensionflag may be used, for example, to provide extension points for a laterversion of the VVC standard.

FIG. 7 illustrates an exemplary LMCS structure according to anembodiment of the present document. The LMCS structure 700 of FIG. 7includes an in-loop mapping part 710 of luma components based onadaptive piecewise linear (adaptive PWL) models and a luma-dependentchroma residual scaling part 720 for chroma components. Thedequantization and inverse transform 711, reconstruction 712, and intraprediction 713 blocks of the in-loop mapping part 710 representprocesses applied in the mapped (reshaped) domain. Loop filters 715,motion compensation or inter prediction 717 blocks of the in-loopmapping part 710, and reconstruction 722, intra prediction 723, motioncompensation or inter prediction 724, loop filters 725 block of thechroma residual scaling part 720 represent processes applied in theoriginal (non-mapped, non-reshaped) domain.

As illustrated in FIG. 7, when LMCS is enabled, at least one of theinverse mapping (reshaping) process 714, a forward mapping (reshaping)process 718, and a chroma scaling process 721 may be applied. Forexample, the inverse mapping process may be applied to a (reconstructed)luma sample (or luma samples or luma sample array) in a reconstructedpicture. The inverse mapping process may be performed based on apiecewise function (inverse) index of a luma sample. The piecewisefunction (inverse) index may identify the piece to which the luma samplebelongs. Output of the inverse mapping process is a modified(reconsturcted) luma sample (or modified luma samples or modified lumasample array). The LMCS may be enabled or disabled at a level of a tilegroup (or slice), picture or higher.

The forward mapping process and/or the chroma scaling process may beapplied to generate the reconstructed picture. A picture may compriseluma samples and chroma samples. A reconstructed picture with lumasamples may be referred to as a reconstructed luma picture, and areconstructed picture with chroma samples may be referred to as areconstructed chroma picture. A combination of the reconstructed lumapicture and the reconstructed chroma picture may be referred to as areconstructed picture. The reconstructed luma picture may be generatedbased on the forward mapping process. For example, if an interprediction is applied to a current block, a forward mapping is appliedto a luma prediction sample derived based on a (reconstructed) lumasample in a reference picture. Because the (reconstructed) luma samplein the reference picture is generated based on the inverse mappingprocess, the forward mapping may be applied to the luma predictionsample thus a mapped (reshaped) luma prediction sample can be derived.The forward mapping process may be performed based on a piecewisefunction index of the luma prediction sample. The piesewise functionindex may be derived based on the value of the luma prediction sample orthe value of the luma sample in the reference picture used for interprediction. If an intra prediction (or an intra block copy (IBC)) isapplied to the current block, the forward mapping is not neccesarybecause the inverse mapping process has not applied to the reconstructedsamples in the current picture yet. A (reconstructed) luma sample in thereconstructed luma picture is generated based on the mapped lumaprediction sample and a corresponding luma residual sample.

The reconstructed chroma picture may be generated based on the chromascaling process. For example, a (reconstructed) chroma sample in thereconstructed chroma picture may be derived based on a chroma predictionsample and a chroma residual sample (c_(res)) in a current block. Thechroma residual sample (c_(res)) is derived based on a (scaled) chromaresidual sample (c_(resScale)) and a chroma residual scaling factor(cScaleInv may be referred to as varScale) for the current block. Thechroma residual scaling factor may be calculated based on reshaped lumaprediction sample values for the current block. For example, the scalingfactor may be calculated based on an average luma value ave (Y′_(pred))of the reshaped luma prediction sample values Y′_(pred). For areference, the (scaled) chroma residual sample derived based on theinverse transform/dequantization may be referred to as c_(resScale), andthe chroma residual sample derived by performing the (inverse) scalingprocess to the (scaled) chroma residual sample may be referred to asc_(res).

FIG. 8 illustrates an LMCS structure according to another embodiment ofthe present document. FIG. 8 is described with reference to FIG. 7.Here, the difference between the LMCS structure of FIG. 8 and the LMCSstructure 700 of FIG. 7 is mainly described. The in-loop mapping partand the luma-dependent chroma residual scaling part of FIG. 8 mayoperate the same as (similarly to) the in-loop mapping part 710 and theluma-dependent chroma residual scaling part 720 of FIG. 7.

Referring to FIG. 8, a chroma residual scaling factor may be derivedbased on luma reconstructed samples. In this case, an average luma value(avgYr) may be obtained (derived) based on the neighboring lumareconstructed samples outside the reconstructed block, not the innerluma reconstructed samples of the reconstructed block, and the chromaresidual scaling factor is derived based on the average luma value(avgYr). Here, the neighboring luma reconstructed samples may beneighboring luma reconstructed samples of the current block, or may beneighboring luma reconstructed samples of virtual pipeline data units(VPDUs) including the current block. For example, when intra predictionis applied to the target block, reconstructed samples may be derivedbased on prediction samples which are derived based on the intraprediction. In the other example, when inter prediction is applied tothe target block, the forward mapping is applied to prediction sampleswhich are derived based on the inter prediction, and reconstructedsamples are generated (derived) based on the reshaped (or forwardmapped) luma prediction samples.

The video/image information signaled through the bitstream may includeLMCS parameters (information on LMCS). LMCS parameters may be configuredas high level syntax (HLS, including slice header syntax) or the like.Detailed description and configuration of the LMCS parameters will bedescribed later. As described above, the syntax tables described in thepresent document (and the following embodiments) may beconfigured/encoded at the encoder end and signaled to the decoder endthrough a bitstream. The decoder may parse/decode information on theLMCS (in the form of syntax components) in the syntax tables. One ormore embodiments to be described below may be combined. The encoder mayencode the current picture based on the information about the LMCS andthe decoder may decode the current picture based on the informationabout the LMCS.

The in-loop mapping of luma components may adjust the dynamic range ofthe input signal by redistributing the codewords across the dynamicrange to improve compression efficiency. For luma mapping, a forwardmapping (reshaping) function (FwdMap) and an inverse mapping (reshaping)function (InvMap) corresponding to the forward mapping function (FwdMap)may be used. The FwdMap function may be signaled using a piece-wiselinear models, for example, the piece-wise linear model may have 16pieces or bins. The pieces may have the equal length. In one example,the InvMap function does not need to be signalled and is instead derivedfrom the FwdMap function. That is, the inverse mapping may be a functionof the forward mapping. For example, the inverse mapping function may bemathematically built as the symmetric function of the forward mapping asreflected by the line y=x.

An in-loop (luma) reshaping may be used to map input luma values(samples) to altered values in the reshaped domain. The reshaped valuesmay be coded and then mapped back into the original (un-mapped,un-reshaped) domain after reconstruction. To compensate for theinteraction between the luma signal and the chroma signal, chromaresidual scaling may be applied. In-loop reshaping is done by specifyinghigh level syntax for the reshaper model. The reshaper model syntax maysignal a piece-wise linear model (PWL model). For example, the reshapermodel syntax may signal a PWL model with 16 bins or pieces of equallengths. A forward lookup table (FwdLUT) and/or an inverse lookup table(InvLUT) may be derived based on the piece-wise linear model. Forexample, the PWL model pre-computes the 1024-entry forward (FwdLUT) andinverse (InvLUT) look up tables (LUT)s. As an example, when the forwardlookup table FwdLUT is derived, the inverse lookup table InvLUT may bederived based on the forward lookup table FwdLUT. The forward lookuptable FwdLUT may map the input luma values Yi to the altered values Yr,and the inverse lookup table InvLUT may map the altered values Yr to thereconstructed values Y′i. The reconstructed values Y′i may be derivedbased on the input luma values Yi.

In one example, the SPS may include the syntax of Table 9 below. Thesyntax of Table 9 may include sps_reshaper_enabled_flag as a toolenabling flag. Here, sps_reshaper_enabled_flag may be used to specifywhether the reshaper is used in a coded video sequence (CVS). That is,sps_reshaper_enabled_flag may be a flag for enabling reshaping in theSPS. In one example, the syntax of Table 9 may be a part of the SPS.

TABLE 9 seq_parameter_set_rbsp( ) { Descriptor  sps_seq_parameter_set_idue(v) ...  sps_reshaper_enabled_flag u(1)  rbsp_trailing_bits( ) }

In one example, semantics on syntax elements sps_seq_parameter_set_idand sps_reshaper_enabled_flag may be as shown in Table 10 below.

TABLE 10 sps_seq_parameter_set_id provides an identifier for the SPS forreference by other syntax elements. sps_reshaper_enabled_flag equal to 1specifies that reshaper is used in the coded video sequence (CVS).sps_reshaper_enabled_flag equal to 0 specifies that reshaper is not usedin the CVS.

In one example, the tile group header or the slice header may includethe syntax of Table 11 or Table 12 below.

TABLE 11 tile_group_header( ) { Descriptor tile_group_pic_parameter_set_id ue(v) ...  if(num_tiles_in_tile_group_minus1 > 0 ) {   offset_len_minus1 ue(v)   For(i

 0; i < num_tiles_in_tile_group_minus1; i

 )    entry_point_offset_minus1[ i ] u(v)  }  if (sps_reshaper_enabled_flag ) {   tile_group_reshaper_model_present_flagu(1)   if ( tile_group_reshaper_model_present_flag )   tile_group_reshaper_model ( )   tile_group_reshaper_enable_flag u(1)  if ( tile_group_reshaper_enable_flag && (!( qtbtt_dual_tree_intra_flag&& tile_group_type == 1 ) ) )   tile_group_reshaper_chroma_residual_scale_flag u(1)  } byte_alignment( ) }

indicates data missing or illegible when filed

TABLE 12 slice_header( ) { Descriptor  slice_pic_parameter_set_id ue(v)...  if( num_tiles_in_slice_minus1 > 0 ) {   offset_len_minus1 ue(v)  for( i

 0; i < num_tiles_in_slice_minus1; i ++ )    entry_point_offset_minus1[i ] u(v)  }  if ( sps_reshaper_enabled_flag ) {  slice_reshaper_model_present_flag u(1)   if(slice_reshaper_model_present_flag )    slice_reshaper_model ( )  slice_reshaper_enable_flag u(1)   if (slice_reshaper_enable_flag &&(!( qtbtt_dual_tree_intra_flag && slice_type

 ] ) ) )    slice_reshaper_chroma_residual_scale_flag u(1)  } byte_alignment( ) }

indicates data missing or illegible when filed

Semantics of syntax elements included in the syntax of Table 11 or Table12 may include, for example, matters disclosed in the following tables.

TABLE 13 tile_group_reshaper_model_present_flag equal to 1 specifiestile group reshaper model ( ) is present in tile group header.tile_group_reshaper_ model_present_flag equal to 0 specifiestile_group_reshaper_model( ) is  not present in tile group header. Whentile_group reshaper model present flag is not present, it is inferred tobe equal to 0. tile_group_reshaper_enabled_flag equal to 1 specifiesthat reshaper is enabled for the current tile group. tile group reshaperenabled flag equal to  0 specifies that reshaper is not enabled for thecurrent tile group. When tile 

 group 

 reshaper 

 enable 

 flag is not present, it is inferred to be equal to 0.tile_group_reshaper_chroma_residual_scale_flag equal to 1 specifies thatchroma residual scaling is enabled for the current tile group.tile_group_ reshaper_chroma_residual_scale flag equal to 0 specifiesthat chroma residual scaling is not enabled for the current tile group.When tile 

group 

 reshaper 

 chroma 

 residual scale 

 flag is not present, it is inferred to be equal to 0.

indicates data missing or illegible when filed

TABLE 14 slice_reshaper_model_present_flag equal to 1 specifiesslice_reshaper_ model( ) is present in slice header.slice_reshaper_model_present_flag equal to 0 specifiesslice_reshaper_model( ) is not present in slice header. Whenslice_reshaper_model_present_flag is not present, it is inferred to beequal to 0. slice_reshaper_enabled_flag equal to 1 specifies thatreshaper is enabled for the current slice. slice_reshaper_enabled_flagequal to 0 specifies that reshaper is not enabled for the current slice.When slice_reshaper_ enable_flag is not present, it is inferred to beequal to 0. slice_reshaper_chroma_residual_scale_flag equal to 1specifies that chroma residual scaling is enabled for the current slice.slice_reshaper_ chroma_residual_scale_flag equal to 0 specifies thatchroma residual scaling is not enabled for the current slice. Whenslice_reshaper_ chroma_residual_scale_flag is not present, it isinferred to be equal to 0.

As one example, once the flag enabling the reshaping (i.e.,sps_reshaper_enabled_flag) is parsed in the SPS, the tile group headermay parse additional data (i.e., information included in Table 13 or 14above) which is used to construct lookup tables (FwdLUT and/or InvLUT).In order to do this, the status of the SPS reshaper flag(sps_reshaper_enabled_flag) may be first checked in the slice header orthe tile group header. When sps_reshaper_enabled_flag is true (or 1), anadditional flag, i.e., tile_group_reshaper_model_present_flag (orslice_reshaper_model_present _flag) may be parsed. The purpose of thetile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be to indicate the presence ofthe reshaping model. For example, whentile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is true (or 1), it may be indicatedthat the reshaper is present for the current tile group (or currentslice). When tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) is false (or 0), it may be indicatedthat the reshaper is not present for the current tile group (or currentslice).

If the reshaper is present and the reshaper is enabled in the currenttile group (or current slice), the reshaper model (i.e.,tile_group_reshaper_model( ) or slice_reshaper_model( )) may beprocessed. Further to this, an additional flag,tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may alsobe parsed. The tile_group_reshaper_enable_flag (or slicereshaper_enable_flag) may indicate whether the reshaping model is usedfor the current tile group (or slice). For example, iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (orfalse), it may be indicated that the reshaping model is not used for thecurrent tile group (or the current slice). Iftile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (ortrue), it may be indicated that the reshaping model is used for thecurrent tile group (or slice).

As one example, tile_group_reshaper_model_present_flag (orslice_reshaper_model_present_flag) may be true (or 1) andtile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may befalse (or 0). This means that the reshaping model is present but notused in the current tile group (or slice). In this case, the reshapingmodel can be used in the future tile groups (or slices). As anotherexample, tile_group_reshaper_enable_flag may be true (or 1) andtile_group_reshaper_model_present_flag may be false (or 0). In such acase, the decoder uses the reshaper from the previous initialization.

When the reshaping model (i.e., tile_group_reshaper_model( ) orslice_reshaper_model( )) and tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) are parsed, it may be determined (evaluated)whether conditions necessary for chroma scaling are present. The aboveconditions includes a condition 1 (the current tile group/slice has notbeen intra-coded) and/or a condition 2 (the current tile group/slice hasnot been partitioned into two separate coding quad tree structures forluma and chroma, i.e. the block structure for The current tilegroup/slice is not a dual tree structure). If the condition 1 and/or thecondition 2 are true and/or tile_group_reshaper_enable_flag (orslice_reshaper_enable_flag) is true (or 1), thentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) may be parsed. Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), itmay be indicated that chroma residual scaling is enabled for the currenttile group (or slice). Whentile_group_reshaper_chroma_residual_scale_flag (orslice_reshaper_chroma_residual_scale_flag) is disabled (if 0 or false),it may be indicated that chroma residual scaling is disabled for thecurrent tile group (or slice).

The purpose of the tile group reshaping model is to parse the data thatwould be necessary to construct the lookup tables (LUTs). These LUTs areconstructed on the idea that the distribution of an allowable range ofluma values can be divided into a plurality of bins (ex. 16 bins) whichcan be represented using a set of 16 PWL system of equations. Therefore,any luma value that lies within a given bin can be mapped to an alteredluma value.

FIG. 9 shows a graph representing an exemplary forward mapping. In FIG.9, five bins are illustrated exemplarily.

Referring to FIG. 9, the x-axis represents input luma values, and they-axis represents altered output luma values. The x-axis is divided into5 bins or slices, each bin of length L. That is, the five bins mapped tothe altered luma values have the same length. The forward lookup table(FwdLUT) may be constructed using data (i.e., reshaper data) availablefrom the tile group header, and thus mapping may be facilitated.

In one embodiment, output pivot points associated with the bin indicesmay be calculated. The output pivot points may set (mark) the minimumand maximum boundaries of the output range of the luma codewordreshaping. The calculation process of the output pivot points may beperformed by computing a piecewise cumulative distribution function(CDF) of the number of codewords. The output pivot range may be slicedbased on the maximum number of bins to be used and the size of thelookup table (FwdLUT or InvLUT). As one example, the output pivot rangemay be sliced based on a product between the maximum number of bins andthe size of the lookup table (size of LUT*maximum number of binindices). For example, if the product between the maximum number of binsand the size of the lookup table is 1024, the output pivot range may besliced into 1024 entries. This serration of the output pivot range maybe performed (applied or achieved) based on (using) a scaling factor. Inone example, the scaling factor may be derived based on Equation 1below.

$\begin{matrix}{{SF} = {{\left( {{y\; 2} - {y\; 1}} \right)*\left( {1\;{\operatorname{<<}{FP\_ PREC}}} \right)} + c}} & \left\lbrack {{Equation}\mspace{20mu} 1} \right\rbrack\end{matrix}$

In Equation 1, SF denotes a scaling factor, and y1 and y2 denote outputpivot points corresponding to each bin. Also, FP_PREC and c may bepredetermined constants. The scaling factor determined based on Equation1 may be referred to as a scaling factor for forward reshaping.

In another embodiment, with respect to inverse reshaping (inversemapping), for a defined range of the bins to be used (i.e., fromreshaper_model_min_bin_idx to reshape_model_max_bin_idx), the inputreshaped pivot points which correspond to the mapped pivot points of theforward LUT and the mapped inverse output pivot points (given by binindex under consideration*number of initial codewords) are fetched. Inanother example, the scaling factor SF may be derived based on Equation2 below.

$\begin{matrix}{{SF} = {\left( {{y\; 2} - {y\; 1}} \right)*{\left( {1\;{\operatorname{<<}{FP\_ PREC}}} \right)/\left( {{x\; 2} - {x\; 1}} \right)}}} & \left\lbrack {{Equation}\mspace{20mu} 2} \right\rbrack\end{matrix}$

In Equation 2, SF denotes a scaling factor, x1 and x2 denote input pivotpoints, and y1 and y2 denote output pivot points corresponding to eachpiece (bin) (output pivot points of the inverse mapping). Here, theinput pivot points may be pivot points mapped based on a forward lookuptable (FwdLUT), and the output pivot points may be pivot pointsinverse-mapped based on an inverse lookup table (InvLUT). Also, FP_PRECmay be a predetermined constant value. FP_PREC of Equation 2 may be thesame as or different from FP_PREC of Equation 1. The scaling factordetermined based on Equation 2 may be referred to as a scaling factorfor inverse reshaping. During inverse reshaping, partitioning of inputpivot points may be performed based on the scaling factor of Equation 2.The scaling factor SF is used to slice the range of input pivot points.Based on the partitioned input pivot points, bin indices in the rangefrom 0 to the minimum bin index (reshaper_model_min_bin_idx) and/or fromthe minimum bin index (reshaper_model_min_bin_idx) to the maximum binindex (reshape_model_max_bin_idx) are assigned the pivot values thatcorrespond to the minimum and maximum bin values.

In one example, the LMCS data (lmcs_data) may be included in the APS.The semantics of APS may be, for example, 32 APSs signaled for coding.

The following tables show syntax and semantics of an exemplary APSaccording to an embodiment of this document.

TABLE 15 adaptation_parameter_set_rbsp( ) { Descriptor adaptation_parameter_set_id u(5)  aps_params_type u(3)  if(aps_params_type −− ALF_APS )   alf data( adaptation_parameter_set_id ) else if( aps_params_type −− LMCS_APS )   lmcs_data( ) aps_extension_flag u(1)  if( aps_extension_flag )   while(more_rbsp_data( ) )    aps_extension_data_flag u(1)  rbsp_trailing_bits() }

Referring to Table 15, type information of APS parameters (e.g.,aps_params_type) may be parsed/signaled in the APS. Type information ofAPS parameters may be parsed/signaled after adaptation_parameter_set_id.

The aps_params_type, ALF_APS, and LMCS APS included in Table 15 abovemay be described according to the following table. That is, according tothe aps_params_type included in Table 15, the types of APS parametersapplied to the APS may be set as shown in Table 3.2 included in Table16.

Referring to Table 16, for example, aps_params_type may be a syntaxelement for classifying types of corresponding APS parameters. The typeof APS parameters may include ALF parameters and LMCS parameters.Referring to Table 16, when the value of type information(aps_params_type) is 0, the name of aps_params_type may be determined asALF_APS (or ALF APS), and the type of APS parameters may be determinedas ALF parameters (the APS parameter may represent the ALF parameters).In this case, the ALF data field (i.e. alf_datao) may be parsed/signaledto the APS. When the value of type information (aps_params_type) is 1,the name of aps_params_type may be determined as LMCS_APS (or LMCS APS),and the type of APS parameters may be determined as LMCS parameters (theAPS parameter may represent the LMCS parameters). In this case, the LMCS(reshaper model, reshaper) data (i.e., lmcs_datao) may beparsed/signaled to the APS.

Tables 17 and/or 18 below show syntax of a reshaper model according toan embodiment. The reshaper model may be referred to as an LMCS model.While here the reshaper model has been exemplarily described as a tilegroup reshaper, the present specification is not necessarily limited tothis embodiment. For example, the reshaper model may be included in theAPS, or the tile group reshaper model may be referred to as a slicereshaper model or LMCS data (LMCS data field). Additionally, the prefix“reshaper_model” or “Rsp” may be used interchangeably with “lmcs”. Forexample, in the following tables and descriptions below,reshaper_model_min_bin_idx, reshaper_model_delta_max_bin_idx,reshaper_model_max_bin_idx, RspCW, RsepDeltaCW may be usedinterchangeably with lmcs_min_bin_idx, lmcs_delta_cs_bin_idx, lmx mixed,lmcs_delta_csDcs_bin_idx, Wlm_idx, respectively.

The LMCS data (lmcs_datao) or the reshaper model (tile group reshaper orslice reshaper) included in Table 15 above may be represented assyntaxes included in the following tables.

TABLE 17 tile_group_reshaper_model ( ) { Descriptor reshaper_model_min_bin_idx ue(v)  reshaper_model_delta_max_bin_idxue(v)  reshaper_model_bin_delta_abs_cw_prec_minus1 ue(v)  for( i

 reshaper_model_min_bin_idx: i

reshaper_model_max_bin_idx: i ++ ) {   reshape_model_bin_delta_abs_CW [i ] u(v)   if ( reshaper_model_bin_delta_abs_CW[ i ] ) > 0 )   reshaper_model_bin_delta_sign_CW_flag[ i ] u(1)  } }

indicates data missing or illegible when filed

TABLE 18 lmcs_data( ) { Descriptor    lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i= lmcs_min_bin_idx; i <=  LmcsMaxBinIdx; i−+ ) {   lmcs_delta_abs_cw [ i] u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_CW_flag[i ] u(1)  }  ... }

The semantics of syntax elements included in the syntax of Table 17and/or Table 18 may include, for example, matters disclosed in thefollowing table.

TABLE 19 reshape_model_min_bin_idx specifies the minimum bin (or piece)index to be used in the reshaper construction process. The value ofreshape model min bin idx shall be in the range of 0 to MaxBinIdx,inclusive. The value of MaxBinIdx shall be equal to 15.reshape_model_delta_max_bin_idx specifies the maximum allowed bin (orpiece) index MaxBinIdx minus the maximum bin index to be used in thereshaper construction process. The value of reshape_model_max_ bin_idxis set equal to MaxBinIdx − reshape model delta max bin idx.reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1 specifies the numberof bits used for the representation of the syntax reshape model bindelta abs CW[ i ]. reshape_model_bin_delta_abs_CW[ i ] specifies theabsolute delta codeword value for the ith bin.reshaper_model_bin_delta_sign_CW_flag[ i ] specifies the sign ofreshape_model_bin_delta_abs_CW[ i ] as follows:  Ifreshape_model_bin_delta_sign_CW_flag[ i ] is equal to 0, the corresponding variable RspDeltaCW[ i ] is a positive value. - Otherwise( reshape 

 model bin delta 

 sign CW 

 flag[ i ]  is not equal to 0 ), the corresponding variable RspDeltaCW[i ]  is a negative value. The variable OrgCW is derived as follows: OrgCW (1 << BitDepthY ) / 16 When reshape 

 model 

 bin 

 delta 

 sign 

 CW 

flag[ i ] is not present, it is inferred to be equal to 0. The variableRspDeltaCW[ i ] (1 − 2*reshape_model_bin_delta_sign_ CW [ i ]) *reshape_model_bin_delta_abs_CW [ i ]; The variable RspCW[ i ] is derivedas following steps: The variable OrgCW is set equal to (1 <<BitDepth_(Y) ) / ( MaxBinIdx − 1). - If reshaper model min bin idx < = i<= reshaper model max bin idx RspCW[ i ] − OrgCW + RspDeltaCW[ i ] Otherwise, RspCW[ i ] 0. The value of RspCW[ i ] shall be in the rangeof (OrgCW>>3) to (OrgCW<<3 − 1), inclusive. The variables InputPivot[ i] with i in the range of 0 to MaxBinIdx + 1, inclusive are derived asfollows:  InputPivot[ i ] = i * OrgCW The variable ReshapePivot[ i ]with i in the range of 0 to MaxBinIdx 

 1, inclusive, the variable ScaleCoef[ i ] and InvScaleCoeff[ i ]with iin the range of 0 to MaxBinIdx , inclusive, are derived as follows: shiftY = 11  ReshapePivot[ 0 ] 0;  for( i = 0, i <= MaxBinIdx ; i+−) {  ReshapePivoti[ i − 1 ] = ReshapePivot[ i ] − RspCW[ i ]   ScaleCoef[ i] = ( RspCW[ i ] * (1 << shiftY) + (1 << (Log2  (OrgCW) − 1))) >>(Log2(OrgCW))   if( RspCW[ i ] == 0 )    InvScaleCoeff[ i ] = 0   else   InvScaleCoeff[ i ] = OrgCW * (1 << shiftY) / RspCW[ i ]  } Thevariable ChromaScaleCoef[ i ] with i in the range of 0 to MaxBinIdx ,inclusive, are derived as follows: if( lmcsCW[ i ]  0 )  ChromaScaleCoeff[ i ] (1 << 11) else   ChromaScaleCoeff[ i ] =InvScaleCoeff[ i ]

indicates data missing or illegible when filed

TABLE 20 lmcs_min_bin_idx specifies the minimum bin (or piece) index tobe used in the luma mapping with chroma scaling construction process.The value of LmcsMaxBinIdx shall be in the range of 0 to MaxBinIdx,inclusive. The value of MaxBinIdx shall be equal to 15.lmcs_delta_max_bin_idx specifies the maximum allowed bin (or piece)index MaxBinIdx minus the maximum bin index to be used in the lumamapping with chroma scaling construction process. The value ofLmcsMaxBinIdx is set equal to 15− lmcs delta max bin idx.lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used forthe representation of the syntax lmcs delta abs cw[ i ].lmcs_delta_abs_cw[ i ] specifies the absolute delta codeword value forthe ith bin. lmcs_delta_sign_cw_flag[ i ] specifies the sign oflmcsDeltaCW[ i ] as follows:  If lmcs_delta_sign_cw_flag[ i ] is equalto 0, the corresponding  variable lmcsDeltaCW CW[ i ] is a positivevalue.  Otherwise ( lmcs_delta_sign_cw_flag[ i ] is not equal to 0 ),the  corresponding variable lmcsDeltaCW[ i ] is a negative value. Whenlmcs 

 delta 

 sign 

 CW 

 flag[ i ] is not present, it is inferred to be equal to 0. The variableOrgCW is derived as follows:  OrgCW = (1 << BitDepthY ) / 16 Thevariable lmcsDeltaCW[ i ], with i = lmcs min bin idx..LmcsMaxBinIdx, isderived as follows:  lmcsDeltaCW[ i ] (1 − 2*lmcs_delta_sign_cw_flag [ i]) *  lmcs_delta_abs_cw [ i ]; The variable lmcsCW[ i ] is derived asfollowing steps:  If lmcs_min_bin_idx <   i < LmcsMaxBinIdx  lmcsCW[ i ]= OrgCW − lmcsDeltaCW[ i ]  Otherwise, lmcsCW[ i ] 0. The value oflmcsCW[ i ] shall be in the range of (OrgCW>>3) to (OrgCW<<3 − 1),inclusive. The variables InputPivot[ i ] with i in the range of 0 toMaxBinIdx − 1, inclusive are derived as follows:  InputPivot[ i ] i *OrgCW The variable LmcsPivot[ i ] with i in the range of 0 to MaxBinIdx− 1, inclusive, the variable ScaleCoeff[ i ] and InvScaleCoeff[ i ] withi in the range of 0 to MaxBinIdx , inclusive, are derived as follows: shiftY 11  ReshapePivot[ 0 ] 0;  for( i = 0; i <= MaxBinIdx ; i−+) {  ReshapePivot[ i − 1 ] = ReshapePivot[ i ] − RspCW[ i ]   ScaleCoef[ i] ( RspCW[ i ] * (1 << shiftY) 

 (1 <<  (Log2(OrgCW) − 1))) >> (Log2(OrgCW))   if ( RspCW[ i ] == 0 )   InvScaleCoeff[ i ] = 0   else    InvScaleCoeff[ i ]  OrgCW * (1 <<shiftY) / RspCW[ i ]  } The variable ChromaScaleCoeff[ i ] with i in therange of 0 to MaxBinIdx , inclusive, are derived as follows: if( lmcsCW[i ] = = 0 )   ChromaScaleCoeff[ i ] = (1 << 11)else                         (7-91)   ChromaScaleCoeff[ i ] =InvScaleCoeff[ i ]

indicates data missing or illegible when filed

The inverse mapping process for the luma sample according to the presentdocument may be described in a form of the standard document as shown inthe table below.

TABLE 21 Inverse mapping process for a luma sample Input to this processis a luma sample lumaSample. Output of this process is a modified lumasample invLumaSample. The value of invLumaSample is derived as follows:  If slice_lmcs_enabled_flag of the slice that contains the luma sample  lumaSample is equal to 1, the following ordered steps apply:  1. Thevariable idxYlnv is derived by invoking the identification  ofpiece-wise function index process for a luma sample as specified in clause 8.8.2.3 with lumaSample as the input and idxYlnv as the output. 2. The variable invSample is derived as follows:    invSample =InputPivot[ idxYInv ] + ( InvScaleCoeff[ IdxYInv ] *     ( lumaSample −LmcsPivot[ idxYInv ] ) − ( 1 << 10 ) ) >> 11  3. The inverse mapped lumasample invLumaSample is derived    as follows:    invLumaSample Clip1Y(invSample ) - Otherwise, invLumaSample is set equal to lumaSample.

Identification of a piecewise function index process for a luma sampleaccording to the present document may be described in a form of thestandard document as shown in the table below. In Table 22, idxYInv maybe referred to as an inverse mapping index, and the inverse mappingindex may be derived based on resconstructed luma samples (lumaSample).

TABLE 22 Identification of piecewise function index process for a lumasample lnput to this process is a luma sample lumaSample. Output of thisprocess is an index idxYlnv identifing the piece to which the lumasample lumaSample belongs. The variable idxYlnv is derived as follows:if ( lumaSample < LmcsPivot[ lmcs_min_bin_idx + 1 ] )  idxYinv = lmcs 

 min 

 bin 

 idx else if( lumaSample >= LmcsPivot[ LmcsMaxBinIdx ] )  idxYinv =LmcsMaxBinIdx else {  for( idxYInv = lmcs min bin idx; idxYInv <LmcsMaxBinIdx;  idxYlnv−− ) {   if( lumaSample < LmcsPivot [ idxYInv + 1] )    break  } }

indicates data missing or illegible when filed

Luma mapping may be performed based on the above-described embodimentsand examples, and the above-described syntax and components includedtherein may be merely exemplary representations, and embodiments in thepresent document are not limited by the above-mentioned tables orequations. Hereinafter, a method for performing chroma residual scaling(scaling for chroma components of residual samples) based on lumamapping is described.

The (luma-dependent) Chroma residual scaling is designed to compensatefor the interaction between the luma signal and its corresponding chromasignals. For example, whether chroma residual scaling is enabled or notis also signalled at the tile group level. In one example, if lumamapping is enabled and if dual tree partition (also known as separatechroma tree) is not applied to the current tile group, an additionalflag is signalled to indicate if the luma-dependent chroma residualscaling is enabled or not. In other example, when luma mapping is notused, or when dual tree partition is used in the current tile group,luma-dependent chroma residual scaling is disabled. In another example,the luma-dependent chroma residual scaling is always disabled for thechroma blocks whose area is less than or equal to 4.

The chroma residual scaling may be based on an average value of acorresponding luma prediction block (a luma component of a predictionblock to which an intra prediction mode and/or an inter prediction modeis applied). Scaling operations at the encoder end and/or the decoderside may be implemented with fixed-point integer arithmetic based onEquation 3 below.

$\begin{matrix}{c^{\prime} = {{{sign}(c)}*\left( {\left( {{{{abs}(c)}*s} + {2{CSCALE\_ FP}{\_ PREC}} - 1} \right)\operatorname{>>}{CSCALE\_ FP\_ PREC}} \right)}} & \left\lbrack {{Equation}\mspace{20mu} 3} \right\rbrack\end{matrix}$

In Equation 3, c′ denotes a scaled chroma residual sample (scaled chromacomponent of a residual sample), c denotes a chroma residual sample(chroma residual sample, chroma component of residual sample), s denotesa chroma residual scaling factor, and CSCALE_FP_PREC denotes a(predefined) constant value to specify precision. For example,CSCALE_FP_PREC may be 11.

FIG. 10 is a flowchart illustrating a method for deriving a chromaresidual scaling index according to an embodiment of the presentdocument. The method in FIG. 10 may be performed based on FIG. 7, andtables, equations, variables, arrays, and functions included in thedescription related to FIG. 7.

In the step S1010, it may be determined whether the prediction mode forthe current block is the intra prediction mode or the inter predictionmode based on the prediction mode information. If the prediction mode isthe intra prediction mode, the current block or prediction samples ofthe current block are considered to be already in the reshaped (mapped)region. If the prediction mode is the inter prediction mode, the currentblock or the prediction samples of the current block are considered tobe in the original (unmapped, non-reshaped) region.

In the step S1020, when the prediction mode is the intra predictionmode, an average of the current block (or luma prediction samples of thecurrent block) may be calculated (derived). That is, the average of thecurrent block in the already reshaped area is calculated directly. Theaverage may also be referred to as an average value, a mean, or a meanvalue.

In the step S1021, when the prediction mode is the inter predictionmode, forward reshaping (forward mapping) may be performed (applied) onthe luma prediction samples of the current block. Through forwardreshaping, luma prediction samples based on the inter prediction modemay be mapped from the original region to the reshaped region. In oneexample, forward reshaping of the luma prediction samples may beperformed based on the reshaping model described with Table 17 and/orTable 18 above.

In the step S1022, an average of the forward reshaped (forward mapped)luma prediction samples may be calculated (derived). That is, anaveraging process for the forward reshaped result may be performed.

In the step S1030, a chroma residual scaling index may be calculated.When the prediction mode is the intra prediction mode, the chromaresidual scaling index may be calculated based on the average of theluma prediction samples. When the prediction mode is the interprediction mode, the chroma residual scaling index may be calculatedbased on an average of forward reshaped luma prediction samples.

In an embodiment, the chroma residual scaling index may be calculatedbased on a for loop syntax. The table below shows an exemplary for loopsyntax for deriving (calculating) the chroma residual scaling index.

TABLE 23 for( idxS − 0. idxFound − 0: idxS <− MaxBinIdx: idxS 

 

  ) {  if( (S < ReshapePivot{ idxS 

 1 ] )  {   idxFound − 1   break:  } }

indicates data missing or illegible when filed

In Table 23, idxS represents the chroma residual scaling index, idxFoundrepresents an index identifying whether the chroma residual scalingindex satisfying the condition of the if statement is obtained, Srepresents a predetermined constant value, and MaxBinIdx represents themaximum allowable bin index. ReshapPivot[idxS+1] (in other words,LmcsPivot[idxS+1]) may be derived based on Tables 19 and/or 20 describedabove.

In an embodiment, the chroma residual scaling factor may be derivedbased on the chroma residual scaling index. Equation 4 is an example forderiving the chroma residual scaling factor.

$\begin{matrix}{s = {{ChromaScaleCoef}\lbrack\;{idxS}\;\rbrack}} & \left\lbrack {{Equation}\mspace{20mu} 4} \right\rbrack\end{matrix}$

In Equation 4, s represents the chroma residual scaling factor, andChromaScaleCoef may be a variable (or array) derived based on Tables 19and/or 20 described above.

As described above, the average luma value of the reference samples maybe obtained, and the chroma residual scaling factor may be derived basedon the average luma value. As described above, the chroma componentresidual sample may be scaled based on the chroma residual scalingfactor, and the chroma component reconstruction sample may be generatedbased on the scaled chroma component residual sample.

In one embodiment of the present document, a signaling structure forefficiently applying the above-described LMCS is proposed. According tothis embodiment of the present document, for example, LMCS data may beincluded in HLS (i.e., an APS), and through the header information(i.e., picture header, slice header) that is a lower level of the APS,an LMCS model (reshaper model) may be adaptively derived by signalingthe ID of the APS, which is referred to the header information. The LMCSmodel may be derived based on LMCS parameters. Also, for example, aplurality of APS IDs may be signaled through the header information, andthrough this, different LMCS models may be applied in units of blockswithin the same picture/slice.

In one embodiment according to the present document, a method forefficiently performing an operation required for LMCS is proposed.According to the semantics described above in Table 19 and/or Table 20,a division operation by the piece length lmcsCW[i] (also noted asRspCW[i] in the present document) is required to deriveInvScaleCoeff[i]. The piece length of the inverse mapping may not bepower of 2, that means the division cannot be performed by bit shifting.

For example, calculating InvScaleCoeff may require up to 16 divisionsper slice. According to Table 19 and/or Table 20 described above, for 10bit coding, the range of lmcsCW[i] is from 8 to 511, so to implement thedivision operation by lmcsCW[i] using the LUT, the size of the LUT mustbe 504. Also, for 12 bit coding, the range of lmcsCW[i] is from 32 to2047, so the LUT size needs be 2016 to implement the division operationby lmcsCW[i] using the LUT. That is, division is expensive in hardwareimplementation, therefore it is desirable to avoid division ifposssible.

In one aspect of this embodiment, lmcsCW[i] may be constriained to bemultiple of a fixed number (or a predetermined number or apre-determined number). Accordingly, a lookup table (LUT) (capacity orsize of the LUT) for the division may be reduced. For example, iflmcsCW[i] becomes multiple of 2, the size of the LUT to replace divisionprocess may be reduced by half.

In another aspect of this embodiment, it is proposed that for codingwith higher internal bit depth coding, on the top of existingconstraints “The value of lmcsCW[i] shall be in the range of (OrgCW>>3)to (OrgCW<<3-1)”, further constrain lmcsCW[i] to be multiple of1<<(BitDepthY-10) if coding bit depth is higher than 10. Here, BitDepthYmay be the luma bit depth. Accordingly, the possible number of lmcsCW[i]would not vary with the coding bitdepth, and the size of LUT needed tocalculate the InvScaleCoeff do not increase for higher coding bitdepth.For example, for 12-bit internal coding bitdepth, limit the values oflmcsCW[i] being multiple of 4, then the LUT to replace division processwill be the same as what is to be used for 10-bit coding. This aspectcan be implemented alone, but it can also be implemented in combinationwith the above-mentioned aspect.

In another aspect of this embodiment, lmcsCW[i] may be constrained to anarrower range. For example, lmcsCW[i] may be constrained within therange from (OrgCW>>1) to (OrgCW<<1)—1. Then for 10 bit coding, the rangeof lmcsCW[i] may be [32, 127], and therefore it only needs a LUT havinga size of 96 to calculate InvScaleCoeff.

In another aspect of the present embodiment, lmcsCW[i] may beapproximated to closest numbers being power of 2, and used that in thereshaper design. Accordingly, the division in the inverse mapping can beperformed (and replaced) by bit shifting.

In one embodiment according to the present document, constraint of theLMCS codeword range is proposed. According to Table 8 described above,the values of the LMCS codewords are in the range from (OrgCW>>3) to(OrgCW<<3)−1. This codword range is too wide. It may result in visualartifact issue when there are large differences between RspCW [i] andOrgCW.

According to one embodiment according to the present document, it isproposed to constrain the codeword of the LMCS PWL mapping to a narrowrange. For example, the range of lmcsCW[i] may be in the range(OrgCW>>1) to (OrgCW<<1)−1.

In one embodiment according to the present document, use of a singlechroma residual scaling factor is proposed for chroma residual scalingin LMCS. The existing method for deriving the chroma residual scalingfactor uses the average value of the corresponding luma block andderives the slope of each piece of the inverse luma mapping as thecorresponding scaling factor. In addition, the process to identify thepiecewise index requires the availability of the corresponding lumablock, it results in latency problem. This is not desirable for hardwareimplementation. According to this embodiment of the present document,scaling in a chroma block may not depend on a luma block value, and itmay not be necessary to identify a piecewise index. Therefore, thechroma residual scaling process in the LMCS can be performed without alatency issue.

In one embodiment according to the present document, the single chromascaling factor may be derived from both the encoder and the decoderbased on the luma LMCS information. When the LMCS luma model isreceived, the chroma residual scaling factor may be updated. Forexample, when the LMCS model is updated, the single chroma residualscaling factor may be updated.

The table below shows an example for obtaining the single chroma scalingfactor according to the present embodiment.

TABLE 24 Sum − 0; for( i − lmcs_min_bin_idx ; <= lmcs_max_bin_idx ; i+−) {  sum −= InvScaleCoeff[ i ] } ChromaScaleCoeff− sum / (lmcs_max_bin_idx- lmcs_min_bin_idx +1);

Referring to Table 24, a single chroma scaling factor (ex.ChromaScaleCoeff or ChromaScaleCoeffSingle) may be obtained by averagingthe inverse luma mapping slopes of all pieces within thelmcs_min_bin_idx and lmcs_max_bin_idx.

FIG. 11 illustrates a linear fitting of pivot points according to anembodiment of the present document. In FIG. 11, pivot points P1, Ps, andP2 are shown. The following embodiments or examples thereof will bedescribed with FIG. 11.

In an example of this embodiment, the single chroma scaling factor maybe obtained based on a linear approximation of the luma PWL mappingbetween the pivot points lmcs_min_bin_idx and lmcs_max_bin_idx+1(LmcsMaxBinIdx+1). That is, the inverse slope of the linear mapping maybe used as the chroma residual scaling factor. For example, the linearline 1 of FIG. 11 may be a straight line connecting the pivot points P1and P2. Referring to FIG. 11, in P1, the input value is x1 and themapped value is 0, and in P2, the input value is x2 and the mapped valueis y2. The inverse slope (inverse scale) of linear line 1 is (x2−x1)/y2,and the single chroma scaling factor ChromaScaleCoeffSingle may becalculated based on the input values and mapped values of pivot pointsP1, P2, and the following equation.

$\begin{matrix}{{ChromaScaleCoeffSingle} = {\left( {{x\; 2} - {x\; 1}} \right)*{\left( {1\;{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/y}\; 2}} & \left\lbrack {{Equation}\mspace{20mu} 5} \right\rbrack\end{matrix}$

In Equation 5, CSCALE_FP_PREC represents a shift factor, for example,CSCALE_FP_PREC may be a predetermined constant value. In one example,CSCALE_FP_PREC may be 11.

In other example according to this embodiment, referring to FIG. 11, theinput value at the pivot point Ps is min_bin_idx+1 and the mapped valueat the pivot point Ps is ys. Accordingly, the inverse slope (inversescale) of the linear line 1 can be calculated as (xs-x1)/ys, and thesingle chroma scaling factor ChromaScaleCoeffSingle may be calculatedbased on the input values and mapped values of the pivot points P1, Ps,and the following equation.

$\begin{matrix}{{ChromaScaleCoeffSingle} = {\left( {{x\; s} - {x\; 1}} \right)*{\left( {1\;{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/{ys}}}} & \left\lbrack {{Equation}\mspace{20mu} 6} \right\rbrack\end{matrix}$

In Equation 6, CSCALE_FP_PREC represents a shift factor (a factor forbit shifting), for example, CSCALE_FP_PREC may be a predeterminedconstant value. In one example, CSCALE_FP_PREC may be 11, and bitshifting for inverse scale may be performed based on CSCALE_FP_PREC.

In another example according to this embodiment, the single chromaresidual scaling factor may be derived based on a linear approximationline. An example for deriving a linear approximation line may include alinear connection of pivot points (i.e., lmcs_min_bin_idx,lmcs_max_bin_idx+1). For example, the linear approximation result may berepresented by codewords of PWL mapping. The mapped value y2 at P2 maybe the sum of the codewords of all bins (fragments), and the differencebetween the input value at P2 and the input value at P1 (x2−x1) isOrgCW*(lmcs_max_bin_idx-lmcs_min_bin_idx+1) (for OrgCW, see Table 19and/or Table 20 above). The table below shows an example of obtainingthe single chroma scaling factor according to the above-describedembodiment.

TABLE 25 Sum = 0; for( i = lmcs 

 min 

 bin 

 idx ; i <= lmcs 

 max 

 bin 

idx ; i++ ) {  sum  

  lmcsCW[ i ] } ChromaScaleCoeffSingle = OrgCW *( lmcs max bin idx-lmcs 

 min 

 bin idx +1)   * (1 << CSCALE_FP_PREC) / sum;

indicates data missing or illegible when filed

Referring to Table 25, the single chroma scaling factor (ex.ChromaScaleCoeffSingle) may be obtained from two pivot points (i.e.,lmcs_min_bin_idx, lmcs_max_bin_idx). For example, the inverse slope oflinear mapping may be used as the chroma scaling factor.

In another example of this embodiment, the single chroma scaling factormay be obtained by linear fitting of pivot points to minimize an error(or mean square error) between the linear fitting and the existing PWLmapping. This example may be more accurate than simply connecting thetwo pivot points at lmcs_min_bin_idx and lmcs_max_bin_idx. There aremany ways to find the optimal linear mapping and such example isdescribed below.

In one example, parameters b1 and b0 of a linear fitting equationy=b1*x+b0 for minimizing the sum of least square error may be calculatedbased on Equations 7 and/or 8 below.

$\begin{matrix}{{b1} = \frac{\sum_{1}^{n}{\left( {x_{i} - \overset{\_}{x}} \right)\left( {y_{i} - \overset{\_}{y}} \right)}}{\sum_{1}^{n}\left( {x_{i} - \overset{\_}{x}} \right)^{2}}} & \left\lbrack {{Equation}\mspace{20mu} 7} \right\rbrack \\{{b\; 0} = {\overset{\_}{y} - {b_{1}\overset{\_}{x}}}} & \left\lbrack {{Equation}\mspace{14mu} 8} \right\rbrack\end{matrix}$

In Equation 7 and 8, x is the original luma values, and y is thereshaped luma values, x and y are the mean of x and y, and xi and yirepresent values of the i-th pivot points.

Referring to FIG. 11, another simple approximation to identify thelinear mapping is given as:

Get the linear line1 by connecting pivot points of PWL mapping atlmcs_min_bin_idx and lmcs_max_bin_idx+1, calculatedlmcs_pivots_linear[i] of this linear line at input values of multiplesof OrgW

Sum up the differences between the pivot points mapped values using thelinear line1 and using PWL mapping.

Get the average difference avgDiff.

Adjust the last pivot point of the linear line according to the averagedifference, e.g., 2*avgDiff

Use the inverse slope of the adjusted linear line as the chroma residualscale.

According to the above-described linear fitting, the chroma scalingfactor (i.e., the inverse slope of forward mapping) may be derived(obtained) based on Equation 9 or 10 below.

$\begin{matrix}{{ChromaScaleCoeffSingle} = {{OrgCW}*{\left( {1\mspace{14mu}{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/{lmcs\_ pivots}}{{\_ linear}\left\lbrack \;{{{lmcs\_ min}{\_ bin}{\_ idx}} + 1}\; \right\rbrack}}} & \left\lbrack {{Equation}\mspace{20mu} 9} \right\rbrack \\{{ChromaScaleCoeffSingle} = {{OrgCW}*\left( {{{lmcs\_ max}{\_ bin}{\_ idx}} - {{lmcs\_ max}{\_ bin}{\_ idx}}\; + 1} \right)*{\left( {1\mspace{14mu}{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/{lmcs\_ pivots}}{{\_ linear}\left\lbrack \;{{{lmcs\_ min}{\_ bin}{\_ idx}} + 1}\; \right\rbrack}}} & \left\lbrack {{Equation}\mspace{14mu} 10} \right\rbrack\end{matrix}$

In the above-described equations, lmcs_pivots_lienar[i] may be mappedvalues of linear mapping. With linear mapping, all pieces of PWL mappingbetween minimum and maximum bin indices may have the same LMCS codeword(lmcsCW). That is, lmcs_pivots_linear[lmcs_min_bin_idx+1] may be thesame as lmcsCW[lmcs_min_bin_idx].

Also, in Equations 9 and 10, CSCALE_FP_PREC represents a shift factor (afactor for bit shifting), for example, CSCALE_FP_PREC may be apredetermined constant value. In one example, CSCALE_FP_PREC may be 11.

With the single chroma residual scaling factor (ChromaScaleCoeffSingle),there is no need to calculate the average of the corresponding lumablock, find the index in the PWL linear mapping to get the chromaresidual scaling factor any more. Accordingly, the efficiency of codingusing chroma residual scaling may be increased. This not only eliminatesthe dependency on corresponding luma block, solves the latency problem,but also reduces the complexity.

The luma dependent chroma residual scaling procedure for semanticsand/or chroma samples related to LMCS data according to the presentembodiment described above may be described in a standard documentformat as shown in the following tables.

TABLE 26 7.4.6.4 Luma mapping with chroma scaling data semanticslmcs_min_bin_idx specifies the minimum bin index used in the lumamapping with chroma scaling construction process. The value of lmcs minbin idx shall be in the range 0 to 15, inclusive. lmcs_delta_max_bin_idxspecifies the delta value between 15 and the maximum bin indexLmcsMaxBinIdx used in the luma mapping with chroma scaling constructionprocess. The value of lmcs_delta_max_bin_ idx shall be in the range of 0to 15, inclusive. The value of LmcsMaxBinIdx is set equal to 15 −lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shall be greater thanor equal to lmcs min bin idx. lmcs_delta_cw_prec_minus1 plus 1 specifiesthe number of bits used for the representation of the syntaxlmcs_delta_abs_cw[ i ]. The value of lmcs_delta_cw_prec_minus1 shall bein the range of 0 to BitDepthY − 2, inclusive. ... The variablediffMaxMinBinIdX, lmcsCWLinear and ChromaScaleCoeffSingle are derived asfollows  diffMaxMinBinIdX− LmcsMaxBinIdx- lmcs min bin idx 

 1:  lmcsCWLinear − (LmcsPivot[LmcsMaxBinIdx 

 1] 

   ( diffMaxMinBinIdX>>1 ))/ diffMaxMinBinIdX:  ChromaScaleCoeffSingle−OrgCW * (1 << 11) /  lmcsCWLinear                    (7-94)

indicates data missing or illegible when filed

TABLE 27 8.7.5.3 Picture reconstruction with luma dependent chromaresidual scaling process for chroma samples Inputs to this process are: a location ( xCurr, yCurr ) of the top-left sample of the current transform block relative to the top-left sample of the current picture. a variable nCurrSw specifying the transform block width, - a variablenCurrSh specifying the transform block height, - a variable tuC 

 fChroma specifying the coded block flag of the  current chromatransform block, - an (nCurrSw) 

 (nCurrSh) array predSamples specifying the  chroma prediction samplesof the current block, - an (nCurrSw) 

 (nCurrSh) array resSamples specifying the chroma  residual samples ofthe current block. Output of this process is a reconstructed chromapicture sample array recSamples. The reconstructed chroma picture samplerecSamples is derived as follows for i 

 0..nCurrSw − 1, j − 0..nCurrSh 1:  If one of the following conditionsis true, recSamples[ 

 Curr 

 i ]  [ yCurr  

  j ] is set equal to Clip1 

 ( predSamples[ i ][ j ] +  resSamples[ i ][ j ] ):- slice_chroma_residual_scale_flag is equal to 0 - nCurrSw * nCurrSh isless than or equal to 4 - tu_c 

 f_c 

 h [ xCurr ][ yCurr ] is equal to 0 and tu_c 

 f_cr  [ xCurr ][ yCurr ] is equal to 0 - Otherwise, the recSamples isderived as follows:   - If tuC 

fChroma is equal to 1, the following applies    resSamples[ i ][ j ] −   Clip3( ( 1 << BitDepth 

  ), ( 1 << BitDepth 

  ) 1,    resSamples[ i ][ j ] )               (8-998)    recSamples[xCurr + i ][ yCurr + j ] − Clip1 

 ( predSamples    [ i ][ j ] −                     (8-999)     Sign(resSamples [ i ][ j ] ) * ( ( Abs( resSamples[ i ][ j ] ) *   ChromaScaleCoeffSingle  

  ( 1 << 10 ) ) >> 11 ) )   - Otherwise (tu_c 

 f is equal to 0), the following applies    recSamples[ xCurr + i ][yCurr + j ] − Clip1 

     (predSamples[ i ][ j ] )              (8-1000)

indicates data missing or illegible when filed

In another embodiment of the present document, the encoder may determineparameters related to the single chroma scaling factor and signal theparameters to the decoder. With signaling, the encoder may utilize otherinformation available at encoder to derive the Chroma scaling factor.This embodiment aims to eliminate the chroma residual scaling latencyproblem.

For example, another example to identify the linear mapping to be usedfor determine Chorom residual scaling factor is given as below:

by connecting pivot points of PWL mapping at lmcs_min_bin_idx andlmcs_max_bin_idx+1, calculated lmcs_pivots_linear[i] of this linear lineat input values of multiples of OrgW

Get weighted sum of the differences between the mapped values of pivotpoints using the linear line1 and those of luma PWL mapping. The weightmay be based on the encoder statistics such as histogram of a bin.

Get the weighted average difference avgDiff.

Adjust the last pivot point of the linear line1 according to theweighted average difference, e.g., 2*avgDiff

Use the inverse slope of the adjusted linear line to calculate thechroma residual scale.

The tables below shows examples of syntaxes for signaling the y valuefor chroma scaling factor derivation.

TABLE 28 lmcs_data( ) { Descriptor    lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i− lmcs_min_bin_idx; i <=  LmcsMaxBinIdx; i−+) {   lmcs_delta_abs_cw[ i ]u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i] u(1)  }  lmcs_chroma_scale u(v) }

In Table 28, the syntax element lmcs_chroma_scale may specify a singlechroma (residual) scaling factor used for LMCS chroma residual scaling(ChromaScaleCoeffSingle=lmcs_chroma_scale). That is, information on thechroma residual scaling factor may be directly signaled, and thesignaled information may be derived as the chroma residual scalingfactor. In other words, the value of the signaled information on thechroma residual scaling factor may be (directly) derived as the value ofthe single chroma residual scaling factor. Here, the syntax elementlmcs_chroma_scale may be signaled together with other LMCS data (i.e., asyntax element related to an absolute value and a sign of the codeword,etc.).

Alternatively, the encoder may signal only necessary parameters toderive the chroma residual scaling factor at decoder. In order to derivethe chroma residual scaling factor at the decoder, it needs an inputvalue x and a mapped value y. Since the x value is the bin length, andis a known number, no need to be signalled. After all, only the y valueneeds to be signaled in order to derive the chroma residual scalingfactor. Here, the y value may be a mapped value of any pivot point inthe linear mapping (i.e., mapped values of P2 or Ps in FIG. 11).

The following tables show examples of signaling mapped values forderiving the chroma residual scaling factor.

TABLE 29 lmcs_data( ) { Descriptor    lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i− lmcs_min_bin_idx; i <−  LmcsMaxBinIdx; i

 ) {   lmcs_delta_abs_cw [ i ] u(v)   if ( lmcs_delta_abs_cw [ i ] ) > 0)    lmcs_delta_sign_cw_flag[ i ] u(1)  }   lmcs_cw_linear u(v) }

indicates data missing or illegible when filed

TABLE 30 lmcs_data( ) { Descriptor    lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_delta_cw_prec_minus1 ue(v)  for ( i− lmcs_min_bin_idx; i <−  LmcsMaxBinIdx; i

 ) {   lmcs_delta_abs_cw [ i ] u(v)   if ( lmcs_delta_abs_cw[ i ] ) > 0)    lmcs_delta_sign_cw_flag[ i ] u(1)  }   lmcs_delta_abs_cw_linearu(v)   if ( lmcs_delta_abs_cw_linear ) > 0 )   lmcs_delta_sign_cw_linear_flag u(1) }

indicates data missing or illegible when filed

One of the syntaxes of Tables 29 and 30 described above may be used tosignal the y value at any linear pivot points specified by the encoderand decoder. That is, the encoder and the decoder may derive the y valueusing the same syntax.

First, an embodiment according to Table 29 is described. In Table 29,lmcs_cw_linear may denote a mapped value at Ps or P2. That is, in theembodiment according to Table 29, a fixed number may be signaled throughlmcs_cw_linear.

In an example according to this embodiment, if lmcs_cw_linear denotes amapped value of one bin (ie, lmcs_pivots_linear[lmcs_min_bin_idx+1] inPs of FIG. 11), the chroma scaling factor may be derived based on thefollowing equation.

$\begin{matrix}{{ChromaScaleCoeffSingle} = {{OrgCW}*{\left( {1\mspace{14mu}{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/{lmcs\_ cw}}{\_ linear}}} & \left\lbrack {{Equation}\mspace{20mu} 11} \right\rbrack\end{matrix}$

In another example according to this embodiment, if lmcs_cw_lineardenotes lmcs_max_bin_idx+1 (i.e. lmcs_pivots_linear[lmcs_max_bin_idx+1]in P2 of FIG. 11), the chroma scaling factor may be derived based on thefollowing equation.

$\begin{matrix}{{ChromaScaleCoeffSingle} = {{OrgCW}*\left( {{{lmcs\_ max}{\_ bin}{\_ idx}} - {{lmcs\_ max}{\_ bin}{\_ idx}}\; + 1} \right)*{\left( {1\mspace{14mu}{\operatorname{<<}{CSCALE\_ FP\_ PREC}}} \right)/{lmcs\_ cw}}{\_ linear}}} & \left\lbrack {{Equation}\mspace{20mu} 12} \right\rbrack\end{matrix}$

In the above-described equations, CSCALE_FP_PREC represents a shiftfactor (a factor for bit shifting), for example, CSCALE_FP_PREC may be apredetermined constant value. In one example, CSCALE_FP_PREC may be 11.

Next, an embodiment according to Table 30 is described. In thisembodiment, lmcs_cw_linear may be signaled as a delta value relative toa fixed number (i.e. lmcs_delta_abs_cw_linear,lmcs_delta_sign_cw_linear_flag). In an example of this embodiment, whenlmcs_cw_linear represents a mapped value inlmcs_pivots_linear[lmcs_min_bin_idx+1] (i.e. Ps of FIG. 11),lmcs_cw_linear_delta and lmcs_cw_linear may be derived based on thefollowing equations.

$\begin{matrix}{{{lmcs\_ cw}{\_ linear}{\_ delta}} = {\left( \;{1 - {2*{lmcs\_ delta}{\_ sign}{\_ cw}{\_ linear}{\_ flag}}}\; \right)*{lmcs\_ delta}{\_ abs}{\_ linear}{\_ cw}}} & \left\lbrack {{Equation}\mspace{20mu} 13} \right\rbrack \\{{{lmcs\_ cw}{\_ linear}} = {{lmcs\_ cw}{\_ linear}{\_ delta}*{OrgCW}}} & \left\lbrack {{Equation}\mspace{14mu} 14} \right\rbrack\end{matrix}$

In another example of this embodiment, when lmcs_cw_linear represents amapped value in lmcs_pivots_linear[lmcs_max_bin_idx+1] (i.e. P2 of FIG.11), lmcs_cw_linear_delta and lmcs_cw_linear may be derived based on thefollowing equations.

$\begin{matrix}{{{lmcs\_ cw}{\_ linear}{\_ delta}} = {\left( \;{1 - {2*{lmcs\_ delta}{\_ sign}{\_ cw}{\_ linear}{\_ flag}}}\; \right)*{lmcs\_ delta}{\_ abs}{\_ linear}{\_ cw}}} & \left\lbrack {{Equation}\mspace{20mu} 15} \right\rbrack \\{{{lmcs\_ cw}{\_ linear}} = {{{lmcs\_ cw}{\_ linear}{\_ delta}} + {{OrgCW}*\left( {{{lmcs\_ max}{\_ bin}{\_ idx}} - {{lmcs\_ max}{\_ bin}{\_ idx}} + 1} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 16} \right\rbrack\end{matrix}$

In the above-described equations, OrgCW may be a value derived based onTable 19 and/or Table 20 described above.

The luma dependent chroma residual scaling procedure for semanticsand/or chroma samples related to LMCS data according to the presentembodiment described above may be described in a standard documentformat as shown in the following tables.

TABLE 31 7.4.6.4 Luma mapping with chroma sealing data semanticslmcs_min_bin_idx specifies the minimum bin index used in the lumamapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_ bin_idx shallbe in the range of 0 to 15. inclusive. The value of LmcsMaxBinIdx is setequal to 15 lmcs 

 delta max bin idx. The value of LmcsMaxBinIdx shall be greater than orequal to lmcs_ min_bin_idx. lmcs_delta_cw_prec_minus1 plus 1 specifiesthe number of bits used for the representation of the syntax lmcs deltaabs cw[ i ] and lmcs delta abs cw linear. The value of lmcs delta cwprec minus1 shall be in the range of 0 to BitDepthY − 2, inclusive.  ...lmcs_delta_abs_cw_linear specifies the absolute delta codeword value forthe ith bin. lmcs_delta_sign_cw_linear_flag specifies the sign of thevariable lmcsDeltaCWLinear as follows:   If lmcs delta sign cw linearflag is equal to 0, lmcsDeltaCWLinear   is a positive value.   Otherwise( lmcs delta sign cw linear flag is not equal to 0 ),  lmcsDeltaCWLinear is a negative value. Whenlmcs_delta_sign_cw_linear_flag is not present, it is inferred to beequal to 0. The variable lmcsDeltaCWLinear, is derived as follows:lmcsDeltaCWLinear − ( 1 − 2 * lmcs_delta_sign_cw_linear_ flag ) *lmcs_delta_ahs_cw_linear lmcsCWLinear − OrgCW − lmcsDeltaCWLinear Thevariable ChromaScaleCoeffSingle is derived as followsChromaScaleCoeffSingle − OrgCW * (1 << 11) / lmcsCWLinear

indicates data missing or illegible when filed

TABLE 32 8.7.5.3 Picture reconstruction with luma dependent chromaresidual scaling process for chroma samples Inputs to this process are: a location ( xCurr, yCurr ) of the top-left sample of the current transform block relative to the top-left sample of the current picture,- a variable nCurrSw specifying the transform block width, - a variablenCurrSh specifying the transform block height, - a variable tuC 

 fChroma specifying the coded block flag of the  current chromatransform block,  an (nCurrSw)x(nCurrSh) array predSamples specifyingthe chroma  prediction samples of the current block.  an(nCurrSw)x(nCurrSh) array resSamples specifying the chroma  residualsamples of the current block Output of this process is a reconstructedchroma picture sample array recSamples. The reconstructed chroma picturesample recSamples is derived as follows for i − 0..nCurrSw 1, j −0..nCurrSh − 1: - If one of the following conditions is true,recSamples[ xCurr − i ]  [ yCurr + j ] is set equal to Clip1 

 ( predSamples[ i ][ j ] 

   resSamples[ i ][ j ] ):  slice 

 chroma residual scale 

 flag is equal to 0  nCurrSw * nCurrSh is less than or equal to 4  tu_c 

 f_cb [ xCurr ][ yCurr ] is equal to 0 and tu c 

 f cr [ xCurr ]  [ yCurr ] is equal to 0  Otherwise, the recSamples isderived as follows:   - If tuC 

 fChroma is equal to 1, the following applies:    resSamples[ i ][ j ] −   Clip3( −( 1 << BitDepth 

  ), (1 << BitDepth 

  ) − 1,    resSamples[ i ][ j ] )            (8-998)    recSamples[  

 Curr 

  i ][ yCurr  

  j ] Clip1 

     ( predSamples[ i ][ j ]  

            (8-999)     Sign( resSamples[ i ][ j ] ) * ( ( Abs(resSamples[ i ][ j ] ) *    ChromaScaleCoeffSingle + ( 1 << 10 ) ) >> 11) )   - Otherwise (tu_c 

 f is equal to 0), the following applies:    recSamples[  

 Curr  

  i ][ yCurr  

  j ] − Clip1 

     (predSamples[ i ][ j ] )            (8-1000)

indicates data missing or illegible when filed

FIG. 12 illustrates one example of linear reshaping (or linearreshaping, linear mapping) according to an embodiment of the presentdocument. That is, in this embodiment, the use of a linear reshaper inLMCS is proposed. For example, this example in FIG. 12 may relate toforward linear reshaping (mapping).

Referring to FIG. 12, the linear reshaper may include two pivot pointsi.e., P1 and P2. P1 and P2 may represent input and mapped values, forexample P1 may be (min_input, 0) and P2 may be (max_input, max_mapped).Here, min-input represents the minimum input value, and max_inputrepresents the maximum input value. Any input value less than or equalto min_input are mapped to 0, any input value larger than max_input aremapped to max_mapped. Any input luma values within the min_input andmax_input are linearly mapped to other values. FIG. 12 shows an exampleof mapping. The pivot points P1, P2 may be determined at the encoder,and a linear fitting may be used to approximate the piecewise linearmapping.

In another embodiment according to the present document, another exampleof a method for signaling the linear reshaper may be proposed. The pivotpoints P1, P2 of the linear reshaper model may be explicitly signaled.The following tables show an example of syntax and semantics forexplicitly signaling the linear reshaper model according to thisexample.

TABLE 33 lmcs_data ( ) { Descriptor  lmcs_min_input ue(v) lmcs_max_input ue(v)  lmcs_max_mapped ue(v)

TABLE 34 lmcs_min_input specifies the input value of the 1st pivotpoint. It has a mapped value of 0. lmcs_max_input is the input value ofthe 2nd pivot point. lmcs_max_mapped is the mapped value at the 2^(nd)pivot points. They may be signaled explicitly using explo golomb code orfixed length code with length depending on BitDepth_(Y). lmcsCWLinearAll − lmcs_max_mapped The variable ScaleCoeffSingle andInvScaleCoeffSingle are derived as follows:  Rounding − (lmcs_max_input−lmcs_min_input)>>1  ScaleCoeffSingle − ( lmcsCWLinearAll * (1 <<FP_PREC) + Rounding)) /   (lmcs_max_input−lmcs_min_input): InvScaleCoeffSingle − (lmcs_max_input− lmcs_min_input)*(1<<FP_PREC) /lmcsCWLinearAll The variable ChromaScaleCoeffSingle is derived asfollows: ChromaScaleCoeffSingle − InvScaleCoeffSingle >> (FP_PREC−CSCALE_FP_PREC)

Referring to Tables 33 and 34, the input value of the first pivot pointmay be derived based on the syntax element lmcs_min_input, and the inputvalue of the second pivot point may be derived based on the syntaxelement lmcs_max_input. The mapped value of the first pivot point may bea predetermined value (a value known to both the encoder and decoder),for example, the mapped value of the first pivot point is 0. The mappedvalue of the second pivot point may be derived based on the syntaxelement lmcs_max_mapped. That is, the linear reshaper model may beexplicitly (directly) signaled based on the information signaled in thesyntax of Table 33.

Alternatively, lmcs_max_input and lmcs_max_mapped may be signaled asdelta values. The following tables show an example of syntax andsemantics for signaling a linear reshaper model as delta values.

TABLE 35 lmcs_data ( ) { Descriptor  lmcs_min_input ue(v) lmcs_max_input_delta ue(v)  lmcs_max_mapped_delta ue(v)

TABLE 36 lmcs_max_input_delta specifies the difference between the inputvalue of the 2nd pivot point to the max luma value (1<<bitdepthY)−1, and lmcs_max_input − ( 1<<bitdepthY)− 1 − lmcs_max_input_delta:lmcs_max_mapped_delta specifies the difference between the mapped valueof the 2nd pivot point to the max luma value (1<<bitdepthY)−1. lmcsCWLinearAll − lmcs_max_mapped −  (1<<bitdepthY)−1 −lmcs_max_mapped_delta The variable ScaleCoeffSingle andInvScaleCoeffSingle are described as follows:  Rounding −(lmcs_max_input− lmcs_min_input)>>1  ScaleCoeffSingle − (lmcsCWLinearAll * (1 << FP_PREC) −   Rounding)) /(lmcs_max_input−lmcs_min_input):  InvScaleCoeffSingle − (lmcs_max_input−lmcs_min_input)*  (1<<FP_PREC) / lmcsCWLinearAll The variableChromaScaleCoeffSingle is derived as follows:  ChromaScaleCoeffSingle −InvScaleCoeffSingle >>  (FP_PREC− CSCALE_FP_PREC)

Referring to Table 36, the input value of the first pivot point may bederived based on the syntax element lmcs_min_input. For example,lmcs_min_input may have a mapped value of 0. lmcs_max_input_delta mayspecify the difference between the input value of the second pivot pointand the maximum luma value (i.e., (1<<bitdepthY)−1).lmcs_max_mapped_delta may specify the difference between the mappedvalue of the second pivot point and the maximum luma value (i.e.,(1<<bitdepthY)−1).

According to an embodiment of the present document, forward mapping forluma prediction samples, inverse mapping for luma reconstructionsamples, and chroma residual scaling may be performed based on theabove-described examples of the linear reshaper. In one example, onlyone inverse scaling factor may be needed for inverse scaling for luma(reconstructed) samples (pixels) in linear reshaper based inversemapping. This is also true for forward mapping and chroma residualscaling. That is, the steps to determine ScaleCoeff[i], InvScaleCoeff[i]and ChromaScaleCoeff[i] with i being the bin index, may be replaced withjust one single factor. Here, one single factor is a fixed pointrepresentation of the (forward) slope or the inverse slope of the linearmapping. In one example, the inverse luma mapping scaling factor(inverse scaling factor in inverse mapping for luma reconstructionsamples) may be derived based on at least one of the followingequations.

$\begin{matrix}{{InvScaleCoeffSingle} = {{OrgCW}/{lmcsCWLinear}}} & \left\lbrack {{Equation}\mspace{20mu} 17} \right\rbrack \\{{InvScaleCoeffSingle} = {{OrgCW}*{\left( {{{Imcs\_ max}{\_ bin}{\_ idx}} - {{lmcs\_ max}{\_ bin}{\_ idx}} + 1} \right)/{lmcsCWLinearAll}}}} & \left\lbrack {{Equation}\mspace{14mu} 18} \right\rbrack \\{{InvScaleCoeffSingle} = {\left( {{{lmcs\_ max}{\_ input}} - {{lmcs\_ min}{\_ input}}}\; \right)/{lmcsCWLinearAll}}} & \left\lbrack {{Equation}\mspace{14mu} 19} \right\rbrack\end{matrix}$

The lmcsCWLinear of Equation 17 may be derived from Table 31 describedabove. lmcsCWLinearALL of Equations 18 and 19 may be derived from atleast one of Tables 33 to 36 described above. In Equation 17 or 18,OrgCW may be derived from Table 19 and/or 20.

The following tables describe equations and syntax (conditionalstatements) indicating a forward mapping process for luma samples (i.e.luma prediction samples) in picture reconstruction. In the followingtables and equations, FP_PREC is a constant value for bit shifting, andmay be a predetermined value. For example, FP_PREC may be 11 or 15.

TABLE 37 idxY − predSamples[ i ][ j ] >> Log2( OrgCW ) PredMapSamples[ i][ j ] − LmcsPivot[ idxY ] + ( ScaleCoeff[ idxY ] * ( predSamples[ i ][j ] − InputPivot[ idxY ] ) −  ( 1 << 10 ) ) >> 11  with i − 0..nCurrSw −1, j − 0..nCurrSh − 1

TABLE 38 if (PredMapSamples[ i ][ j ] <− lmcs_min_input) PredMapSamples[ i ][ j ]−0 else if (PredMapSamples[ i ][ j ] >−lmcs_max_input)  PredMapSamples[ i ][ j ]−lmcs_max_mapped else PredMapSamples[ i ][ j ] − ( ScaleCoeffSingle * predSamples[ i ][ j ] −  ( 1 <<( FP_PREC−1) ) ) >> FP_PREC

Table 37 may be for deriving forward mapped luma samples in the lumamapping process based on Tables 17 to 20 described above. That is, Table37 may be described together with Tables 19 and 20. In Table 37, theforward mapped luma (prediction) samples PredMapPSamples[i][j] as outputcan be derived from luma (prediction) samples predSamples[i][j] asinput. idxY of Table 37 may be referred to as a (forward) mapping index,and the mapping index may be derived based on prediction luma samples.

Table 38 may be for deriving forward mapped luma samples in the linearreshaper based luma mapping. For example, lmcs_min_input,lmcs_max_input, lmcs_max_mapped, and ScaleCoeffSingle of Table 38 may bederived by at least one of Tables 33 to 36. In Table 38, in a case of‘lmcs_min_input<predSamples[i][j]<lmcs_max_input’, forward mapped luma(prediction) samples PredMapSamples[i][j] may be derived as output fromthe input, luma (prediction) samples predSamples[i][j]. In comparisonbetween Table 37 and Table 38, the change from the existing LMCSaccording to the application of the linear reshaper can be seen from theperspective of forward mapping.

The following equations and tables describe an inverse mapping processfor luma samples (i.e. luma reconstruction samples). In the followingequations and tables, ‘lumaSample’ as an input may be a lumareconstruction sample before inverse mapping (before modification).‘invSample’ as output may be an inverse mapped (modified) lumareconstruction sample. In other cases, the clipped invSample may bereferred to as a modified luma reconstruction sample.

$\begin{matrix}{\left. {{invSample} = {{{InputPivot}\lbrack\;{idxYInv}\;\rbrack} + \left( \;{{{{InvScaleCoeff}\lbrack\;{idxYInv}\;\rbrack}*\left( \;{{lumaSample} - {{LmcsPivot}\lbrack\;{idxYInv}\;\rbrack}}\; \right)} + \left( \;{{1\mspace{11mu}{\operatorname{<<}(}\mspace{11mu}{FP\_ PREC}} - 1} \right)}\; \right)}}\; \right)\operatorname{>>}\;{FP\_ PREC}} & \left\lbrack {{Equation}\mspace{20mu} 20} \right\rbrack \\\left. {{invSample} = {{{lmcs\_ min}{\_ input}} + \left( \;{{{InvScaleCoeffSingle}*\left( \;{{lumaSample} - {{lmcs\_ min}{\_ input}}} \right)} + \left( \;{{1\mspace{11mu}{\operatorname{<<}(}\mspace{11mu}{FP\_ PREC}} - 1} \right)}\; \right)}}\; \right) & \left\lbrack {{Equation}\mspace{14mu} 21} \right\rbrack\end{matrix}$

TABLE 39 Use single inverse scale factor, derive the invserse lumasample as a whole linear piece.  lumaSample − Clip3(0,LmcsPivot[LmcsMaxBinIdx−1], lumaSample) // Clip3(min, max, val) invSample − InputPivot[ lmcs_min_bin_idx ] − ( InvScaleCoeffSingle *  ( lumaSample − LmcsPivot[ lmcs_min_bin_idx ] ) − ( 1 << ( FP_PREC−1) )) >> FP_PREC where Clip3(min, max, val) denotes to clip the input vlaueval within range min and max

TABLE 40 If the pivot point are represented by (lmcs_min_input,0)(lmcs_max_input, lmcs_max_mapped)  lumaSample − Clip3(0,lmcs_max_mapped, lumaSample) // Clip3(min, max, val) invSample −lmcs_min_input − ( InvScaleCoeffSingle * lumaSample + ( 1 << (FP_PREC−1)) ) >> FP_PREC

Equation 21 may be for deriving inverse mapped luma samples in lumamapping according to this document. In Equation 20, the index idxInv maybe derived based on Tables 50, 51, or 52 to be described later.

Equation 21 may be for deriving inverse mapped luma samples from lumamapping according to the application of the linear reshaper. Forexample, lmcs_min_input of Equation 21 may be derived from at least oneof Tables 33 to 36. Through the comparison between Equation 20 andEquation 21, the change from the existing LMCS according to theapplication of the linear reshaper can be seen from the perspective offorward mapping.

Table 39 may include an example of equations for deriving inverse mappedluma samples in luma mapping. For example, the index idxInv may bederived based on Tables 50, 51, or 52 to be described later.

Table 40 may include other examples of equations for deriving inversemapped luma samples in luma mapping. For example, lmcs_min_input and/orlmcs_max_mapped of Table 40 may be derived by at least one of Tables 33to 36, and/or InvScaleCoeffSingle of Table 40 is at least of Tables 33to 36, and/or Equations 17 to 19 can be derived by one.

Based on the above-described examples of the linear reshaper, thepiecewise index identification process may be omitted. That is, in thepresent examples, since there is only one piece having valid reshapedluma pixels, the piecewise index identification process used for inverseluma mapping and chroma residual scaling can be removed. Accordingly,the complexity of inverse luma mapping may be reduced. In addition,latency problems caused by depending on luma piecewise indexidentification during chroma residual scaling can be eliminated.

According to the embodiment of the use of the linear reshaper describedabove, the following advantages may be provided for LMCS: i) It ispossible to simplify the encoder reshaper design, preventing possibleartifact by abrupt changes between the piecewise linear pieces ii) Thedecoder inverse mapping process, in which the piecewise indexidentification process can be removed, can be simplified by eliminatingthe piecewise index identification process iii) By removing thepiecewise index identification process, it is possible to remove thelatency problem in the chroma residual scaling caused by depending onthe corresponding luma blocks iv) It is possible to reduce overhead ofsignaling, and make frequent update of reshaper more feasible v) Formany places that used to require of a loop of 16 pieces, the loop can beelimated. For example, to derive InvScaleCoeff[i], the number ofdivision operations by lmcsCW[i] can be reduced to 1.

In another embodiment according to the present document, an LMCS basedon flexible bins is proposed. Here, the flexible bins may refers to thenumber of bins not fixed to a predetermined (predefined, specific)number. In the existing embodiment, the number of bins in the LMCS isfixed to 16, and the 16 bins are equally distributed for input samplevalues. In this embodiment, the flexible number of bins is proposed andthe pieces (bins) may not be equally distributed in terms of theoriginal pixel values.

The following tables exemplarily show syntax for LMCS data (data field)according to this embodiment and semantics for syntax elements includedtherein.

TABLE 41 lmcs_data ( ) { Descriptor    lmcs_num_bins_minus1 ue(v)   for( i = 0; i <= lmcs_num_bins; i+−) { u(v)     lmcs_delta_input_cw[ i ]u(v)    lmcs_delta_mapped_cw[ i ] u(v)   }

TABLE 42 lmcs_num_bins_minus1 plus 1 specifies the number of bins.lmcs_num_bins shall be in the range of 1 and 1<<BitDepth_(Y) −1.lmcs_num_bins or lmcs_num_bins_minus1 may be restricted to multiple ofpower of 2. or log of 2 to reduce the number of bits used for signaling.lmcs_num_bins − lmcs_num_bins_minus1 + 1 lmcs_delta_input_cw[ i ]specifies the delta input value of the ith pivot point relative theprevious pivot point. lmcs_delta_input_cw[ i ] >− 0.lmcs_delta_mapped_cw[ i ] specifies the delta mapped value of the ithpivot point relative the previous pivot point. lmcs_delta_mapped_cw[ i] >− 0 The variable LmcsPivot_input[ i ] and LmcsPivot_mapped[ i ] withi − 0.. lmcs_num_bins [ i ] LmcsPivot_input[ 0 ] − 0; LmcsPivot_mapped[0 ] − 0: for( i − 0: i <− lmcs_num_bins; i−− ) {  LmcsPivot_input[ i − 1] − LmcsPivot_input[ i ] |  lmcs_delta_input_cw[ i ]:  LmcsPivot_mapped[i − 1 ] − LmcsPivot_mapped[ i ] |  lmcs_delta_mapped_cw[ i ]: }LmcsPivot_input[ lmcs_num_bins+][ shall equal to (1 << BitDepth_(Y) ) −1 . l.e. sum of all lmcs_delta_input_cw [ i ] shall equal to ( 1<<BitDepth_(Y) ) 1, therefore the last lmcs_delta_mapped_cw[ i ] can beinferred without signaling. LmcsPivot_mapped[ lmcs_num_bins−1]. l.e. sumof all lmcs_delta_mapped_cw[ i ] shall be not greater than (1 <<BitDepth_(Y) ) 1.

Referring to Table 41, information on the number of binslmcs_num_bins_minus1 may be signaled. Referring to Table 42,lmcs_num_bins_minus1+1 may be equal to the number of bins, from whichthe number of bins may be within a range from 1 to (1<<BitDepthY)−1. Forexample, lmcs_num_bins_minus1 or lmcs_num_bins_minus1+1 may be multipleof power of 2.

In the embodiment described together with tables 41 and 42, the numberof pivot points can be derived based on lmcs_num_bins_minus1(information on the number of bins), regardless of whether the reshaperis linear or not (signaling of lmcs_num_bins_minus1), and input valuesand mapped values of pivot points (LmesPivot_input[i],LmesPivot_mapped[i]) may be derived based on the summation of signaledcodeword values (lmes_delta input_cw[i], lmcs_delta mapped_cw[i]) (here,the initial The input value LmcsPivot_input[0] and the initial outputvalue LmcsPivot mapped[0] are 0).

FIG. 13 shows an example of linear forward mapping in an embodiment ofthe present document. FIG. 14 shows an example of inverse forwardmapping in an embodiment of the present document.

In the embodiment according to FIG. 13 and FIG. 14, a method forsupporting both regular LMCS and linear LMCS is proposed. In an exampleaccording to this embodiment, a regular LMCS and/or a linear LMCS may beindicated based on the syntax element lmcs_is_linear. In the encoder,after the linear LMCS line is determined, the mapped value (i.e., themapped value in pL of FIG. 13 and FIG. 14) can be divided into equalpieces (i.e., LmcsMaxBinIdx−lmcs_mm_bin_idx+1). The codeword in the binLmcsMaxBinIdx may be signaled using the syntaxes for the umcs data orreshaper mode described above.

The following tables exemplarily show syntax for LMCS data (data field)and semantics for syntax elements included therein according to anexample of this embodiment.

TABLE 43 lmcs_data ( ) { Descriptor     lmcs_min_bin_idx ue(v)  lmcs_delta_max_bin_idx ue(v)   lmcs_is_linear_flag u(1)  lmcs_delta_cw_prec_minus1 ue(v)  If (lmcs_is_linear_flag){   lmcs_bin_delta_abs_CW_linear u(v)    if(lmcs_bin_delta_abs_CW_linear) > 0 )    lmcs_bin_delta_sign_CW_linear_flag u(1)   }  else {   for ( i =lmcs_min_bin_idx;   i <= LmcsMaxBinIdx; i++ ) {    lmcs_delta_abs_cw[ i] u(v)    if ( lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i ] u(1)   }  } }

TABLE 44 lmcs_min_bin_idx specifies the minimum bin index used in theluma mapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in therange of 0 to 15, inclusive. The value of LmcMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shallbe greater than or equal to lmcs_min_bin_idx. lmcs_is_linear_flag equalto 1 specifies that LMCS model is a linear model, otherwise, it is aregular 16-piece PWL model. lmcs_delta_cw_prec_minus1 plus 1 specifiesthe number of bits used for the representation of the syntaxlmcs_delta_abs_cw[ i ] if lmcs_is_linear_flag is false and specifies thenumber of bits used for the representation of thelmcs_bin_delta_abs_CW_linear if lmcs_is_linear_flag is true. The valueof lmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY 2,inclusive. lmcs_delta_abs_cw_linear specifies the absolute deltacodeword value of one bin of linear mapping.lmcs_delta_sign_cw_linear_flag specifies the sign of the variablelmcsDeltaCWLinear as follows: - If lmcs_delta_sign_cw_linear_flag isequal to 0, lmcsDeltaCWLinear is a positive value. - Otherwise (lmcs_delta_sign_cw_linear_flag is not equal to 0 ), lmcsDeltaCWLinear isa negative value. When lmcs_delta_sign_cw_linear_flag is not present, itis inferred to be equal to 0. lmcs_delta_abs_cw[ i ] specifies theabsolute delta codeword value for the ith bin. lmcs_delta_sign_cw_flag[i ] specifies the sign of the variable lmcsDeltaCW[ i ] as follows: Iflmcs_delta_sign_cw_flag[ i ] is equal to 0, lmcsDeltaCW[ i ] is apositive value. Otherwise ( lmcs_delta_sign_cw_flag[ i ] is not equal to0 ), lmcsDeltaCW[ i ] is a negative value. When lmcs_delta_sign_cw_flag[i ] is not present, it is inferred to be equal to 0.  The variable OrgCWis derived as follows:    OrgCW = (1 << BitDepthY ) / 16 (7-88)  Thevariable lmcsDeltaCW[ i ], with i lmcs_min_bin_idx..  LmcsMaxBinIdx, isderived as follows:   if lmcs_is_linear_flag is true,    lmcsDeltaCW[ i] =     ( 1 − 2 * lmcs_delta_sign_cw_linear_flag ) *    lmcs_delta_abs_cw_linear   else    lmcsDeltaCW[ i ]    ( 1 − 2 *lmcs_delta_sign_cw_flag[ i ] ) * (7-89)    lmcs_delta_abs_cw[ i ]  Thevariable lmcsCW[ i ] is derived as follows:  -  For i

 0..lmcs_min_bin_idx 1, lmcsCW[ i ] is set equal 0.   For ilmcs_min_bin_idx..LmcsMaxBinIdx,   the following applies:    lmcsCW[ i ]OrgCW − lmcsDeltaCW[ i ] (7-90)   The value of lmcsCW[ i ] shall be inthe range of   (OrgCW>>3) to (OrgCW<<3 − 1), inclusive.  -  For i

 LmcsMaxBinIdx

 1..15, lmcsCW[ i ]     is set equal 0.  It is a requirement ofbitstream conformance that  the following condition is true:    Σ_(i=0)¹⁵lmcsCW[ i ] <= (1 << BitDepthY ) − 1 (7-91)

indicates data missing or illegible when filed

The following tables exemplarily show syntax for LMCS data (data field)and semantics for syntax elements included therein according to anotherexample of this embodiment.

TABLE 45 lmcs_data ( ) { Descriptor    lmcs_min_bin_idx ue(v) lmcs_delta_max_bin_idx ue(v)  lmcs_is_linear_flag u(1) lmcs_delta_cw_prec_minus1 ue(v)  for ( i =lmcs_min_bin_idx;  i <=LmcsMaxBinEnd; i++ ) {   lmcs_delta_abs_cw[ i ] u(v)   if (lmcs_delta_abs_cw[ i ] ) > 0 )    lmcs_delta_sign_cw_flag[ i ] u(1)  } }

TABLE 46 lmcs_min_bin_idx specifies the minimum bin index used in theluma mapping with chroma scaling construction process. The value oflmcs_min_bin_idx shall be in the range of 0 to 15, inclusive.lmcs_delta_max_bin_idx specifies the delta value between 15 and themaximum bin index LmcsMaxBinIdx used in the luma mapping with chromascaling construction process. The value of lmcs_delta_max_bin_idx shallbe in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is setequal to 15 − lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shallbe greater than or equal to lmcs_min_bin_idx. lmcs_is_linear_flag equalto 1 specifies that LMCS model is linear model, otherwise, equal to 0specifies that LMCS model is a regular 16-piece PWL model.lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used forthe representation of the syntax lmcs_delta_abs_cw[ i ]. The value oflmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY 2,inclusive. Variables LmcsMaxBinEnd is derived as below:  if(lmcs_is_linear_flag==1)    LmcsMaxBinEnd = lmcs_min_bin_idx  else  LmcsMaxBinEnd = LmcsMaxBinIdx lmcs_delta_abs_cw[ i ] specifies theabsolute delta codeword value for the ith bin. lmcs_delta_sign_cw_flag[i ] specifies the sign of the variable lmcsDeltaCW[ i ] as follows: - Iflmcs_delta_sign_cw_flag[ i ] is equal to 0, lmcsDeltaCW[ i ] is apositive value. - Otherwise ( lmcs_delta_sign_cw_flag[ i ] is not equalto 0 ), lmcsDeltaCW[ i ] is a negative  value. Whenlmcs_delta_sign_cw_flag[ i ] is not present, it is inferred to be equalto 0. When lmcs_is_linear_flag is true, the start index and end index ofthe loop to receive lmcs_delta_abs_cw[ i ] are the same, therefore theloop is only entered once, only one set of lmcs_delta_abs_cw[ i ] andlmcs_delta_sign_cw[ i ] is received, and is stored inlmcs_delta_abs_cw[lmcs_min_bin_idx ] andlmcs_delta_sign_cw[lmcs_min_bin_idx].     The variable OrgCW is derivedas follows:       OrgCW = (1 << BitDepthY ) / 16 (7-88)     The variablelmcsDeltaCW[ i ], with i lmcs_min_bin_idx..LmcsMaxBinIdx,     is derivedas follows:      if lmcs_is_linear_flag is true,       lmcsDeltaCW[ i ]=      ( 1 − 2 * lmcs_delta_sign_cw_flag[lmcs_min_bin_idx] ) *     lmcs_delta_abs_cw[lmcs_min_bin_idx]      else     lmcsDeltaCW[ i ]= ( 1 − 2 * lmcs_delta_sign_cw_flag[ i ] ) * lmcs_delta_abs_cw[ i ](7-89)

Referring to Tables 43 to 46, when lmcs_is_linear_flag is true, all thelmcsDeltaCW[i] between lmcs_min_bin_idx and LmcsMaxBinIdx may have thesame values. That is, lmcsCW[i] of all pieces between lmcs_min_bin_idxand LmcsMaxBinIdx, may have the same values. The scale and inverse scaleand chroma scale of all pieces between lmcs_min_bin_idx andlmcsMaxBinIdx may be the same. Then if the linear reshaper is true, thenthere is no need to derive the piece index, it can use scale, inversescale from just one of the pieces.

The following table exemplarily shows the identification process of thepiecewise index according to the present embodiment.

TABLE 47 Identification of piecewise function index process for a lumasample  if lmcs_is_linear_flag is true,    idxYInv − lmcs_min_bin_idx else  {   /* Use existing piecewise index identification process */  }

According to another embodiment of the present document, the applicationof regular 16-piece PWL LMCS and linear LMCS may be dependent on higherlevel syntax (i.e., sequence level).

The following tables exemplarily show the syntax for the SPS accordingto the present embodiment and the semantics for the syntax elementsincluded therein.

TABLE 48 seq_parameter_set_rbsp( ) { Descriptor .... sps_linear_lmcs_enabled_flag u(1) ...

TABLE 49 sps_linear_lmcs_enabled_flag equal to 0 specifies that thelinear lmcs is disabled and only regular LMCS is enabled.sps_linear_lmcs_enabled_flag equal to 1 specifies that the linear lmcsis enabled and regular LMCS is disabled.

Referring to Tables 48 and 49, enabling of the regular LMCS and/or thelinear LMCS may be determined (signaled) by a syntax element included inthe SPS. Referring to Table 35, based on the syntax elementsps_linear_lmcs_enabled_flag, one of a regular LMCS or a linear LMCS maybe used in units of a sequence.

In addition, whether to only enable the linear LMCS, or the regular LMCSor both may also be dependent on profile level. In one example, for aspecific profile (i.e., SDR profile), it may only allow the linear LMCS,and for another profile (i.e., HDR profile), it may only allow theregular LMCS, and for another profile, it may allow both regular LMCSand/or linear LMCS.

According to another embodiment of the present document, the LMCSpiecewise index identification process may be used in inverse lumamapping and chroma residual scaling. In this embodiment, theidentification process of the piecewise index may be used for blockswith chroma residual scaling enabled, and also invoked for all lumasamples in reshaped (mapped) domain. The present embodiment aims to keepits complexity low.

The following table shows the identification process (derivationprocess) of the existing piecewise function index.

TABLE 50 Identification of piecewise function index process for a lumasample  if ( lumaSample < LmcsPivot[ lmcs_min_bin_idx + 1 ] )   idxYInv− lmcs_min_bin_idx  else if ( lumaSample >− LmcsPivot[ LmcsMaxBinIdx ] )  idxYInv − LmcsMaxBinIdx  else { (8-1003)   for( idxYInv −lmcs_min_bin_idx: idxYInv <   LmcsMaxBinIdx: idxYInv−+ ) {    if(lumaSample < LmcsPivot [ idxYInv − 1 ] )     break   }  }

In an example, in the piecewise index identification process, inputsamples may be classified into at least two categories. For example, theinput samples may be classified into three categories, first, second,and third categories. For example, the first category may representsamples (values thereof) less than LmcsPivot[lmcs_min_bin_idx+1], andthe second category may represent samples (values thereof) greater thanor equal to (values thereof) LmcsPivot[LmcsMaxBinIdx], The thirdcategory may indicate (values of) samples betweenLmcsPivot[lmcs_min_bin_idx+1] and LmcsPivot[LmcsMaxBinIdx].

In this embodiment, it is proposed to optimize the identificationprocess by eliminating the categories classifications. This is becausethe input to the piecewise index identification process are luma valuesin reshaped (mapped) domain, there should be no values beyond the mappedvalues at the pivot points lmcs_min bin_idx and LmcsMaxBinIdx+1.Accordingly, the conditional process to classify samples into categoriesin the existing piecewise index identification process is unnecesary.For more details, specific examples will be described below with tables.

In an example according to this embodiment, the identification processincluded in Table 50 may be replaced with one of Tables 51 or 52 below.Referring to Tables 51 and 52, the first two categories of Table 50 maybe removed, and for the last category, the boundary value (the secondboundary value or the ending point) in the iterative for loop is changedfrom LmcsMaxBinIdx to LmcsMaxBinIdx+1. That is, the identificationprocess may be simplified and the complexity for the piecewise indexderivation may be reduced. Accordingly, LMCS-related coding can beefficiently performed according to the present embodiment.

TABLE 51   for( idxYInv − lmcs_min_bin_idx; idxYInv <− LmcsMaxBinIdx;idxYInv−+ ) {  if( lumaSample < LmcsPivot [ idxYInv

 1 ] )   break }

indicates data missing or illegible when filed

TABLE 52   for( idxYInv − lmcs_min_bin_idx; idxYInv < LmcsMaxBinIdx+1;idxYInv+− ) {  if( lumaSample < LmcsPivot [ idxYInv

 1 ] )   break }

indicates data missing or illegible when filed

Referring to Table 51, comparison process corresponding to the conditionof the if statement (an equation corresponding to the condition of theif statement) may be iteratively performed on all of the bin indicesfrom the minimum bin index to the maximum bin index. The bin index, inthe case where the equation corresponding to the condition of the ifstatement is true, may be derived as the inverse mapping index forinverse luma mapping (or the inverse scaling index for chroma residualscaling). Based on the inverse mapping index, modified reconstructedluma samples (or scaled chroma residual samples) may be derived.

According to Table 51, a problem that may be generated in the procedureof identifying an (inverse) piecewise function index can be solved. Dueto the embodiment of Table 51, calculation bugs can be removed and/oroverlapping calculation processes that are generated due to overlappingboundary conditions can be omitted. If the present document does notfollow the embodiment of Table 51, the mapping value, which is used inLMCS about the current block may exceed (come out of)LmcsPivot[idxYInv+1] in the for syntax (loop syntax) for identificationof the piecewise index (inverse mapping index). According to the presentembodiment, a mapping value within an appropriate range that is used inLMCS about the current block may be used in the index identificationprocedure.

According to an embodiment of the present document, explicit chromaresidual scaling factor signaling or implicit (indirect) chroma residualscaling factor derivation may be applied (performed). Flag informationshowing (indicating) whether explicit chroma residual scaling factorsignaling or implicit chroma residual scaling factor derivation isapplied may be included in an SPS. That is, the flag information may bean SPS level flag. In the present embodiment, an LMCS model (LMCS datafiled) may be signaled in an APS.

In an embodiment, in explicit chroma residual scaling factor derivation,information about (a) value(s) of a chroma residual scaling factor maybe explicitly signaled from an encoder to a decoder. For example, theinformation about (a) value(s) of a chroma residual scaling factor maybe signaled in the unit of n×m blocks (CTUs). Here, n and/or m may be apositive integer defined in advance. The value(s) of n and/or m may besignaled at a slice header. Accordingly, it may be possible to controlgranularity (about signaling and application) of the value(s) of achroma residual scaling factor (into the unit of CTU (block) from theunit of slice). In explicit signaling, the value(s) of a chroma residualscaling factor may be decoded in the unit of n×m CTUs. Chroma residualscaling may be applied to use the value(s) of a chroma residual scalingfactor decoded for all chroma blocks pertaining to the region of n×mCTUs. For example, the coding procedure of the value(s) of a chromaresidual scaling factor may be substantially the same as or similar tothe coding procedure of a delta quantization parameter (QP).

In one example, in implicit chroma residual scaling factor derivation, achroma residual scaling factor may be derived on the basis of lumacomponents for the current block (ex. a block pertaining to a VPDUlevel) (luma samples for the current block). In implicit derivation, achroma residual scaling factor for each block (VPDU) may be derived onthe basis of the (values of) luma samples included in previously codedadjacent blocks (VPDUs). In order to derive a chroma residual scalingfactor for each block, left adjacent reconstructed luma samples and/ortop adjacent reconstructed luma samples may be used.

FIG. 15 is a view illustrating an LMCS procedure according to anembodiment of the present document. Derivation of an implicit chromaresidual scaling factor may be described with reference to FIG. 15.

Referring to FIG. 15, luma samples that are used to derive a chromaresidual scaling factor may include M left adjacent luma samples and Mtop luma samples. In one example, M left adjacent luma samples and/or Mtop luma samples may be close to the position of the left-top sample.For example, M may be 16. As shown in FIG. 16, an average value of the Mleft adjacent luma samples and M top luma samples may be derived(calculated) to derive a chroma residual scaling factor. Derivation of achroma residual scaling factor based on the average value of lumasamples was described in detail with reference to FIG. 11 and relateddescription. A chroma residual scaling factor derived for one block(VPDU) may be used for all chroma blocks (samples) included in the oneblock. In accordance with the above-described embodiment, coding for thecurrent block may be performed without no more additional dependencebetween luma samples and chroma samples.

When the number of top adjacent samples is smaller than M for the blocksincluded in the rightmost column at a picture boundary, picture boundarysamples may be padded up to M. When the number of left adjacent samplesis smaller than M for the blocks included in the lowermost row, pictureboundary samples may be padded up to M. Accordingly, the number ofsamples that are averaged may always be 2M (ex. M is a multiple of 2),and thus, division may be omitted. For the blocks included in theleftmost column, only M top adjacent samples may be used to derive anaverage value. Further, for the blocks included in the uppermost row,only M left adjacent samples may be used to derive an average value. Inaddition, for the left-top block of the picture, the average value maybe set as a value determined in advance, and for example, the valuedetermined in advance may be 1<<(bitdepth-1).

The following table shows exemplary syntaxes of an SPS according to thepresent embodiment.

TABLE 53 seq_parameter_set_rbsp( ) { Descriptor sps_decoding_parameter_set_id u(4)  sps_video_parameter_set_id u(4)  if( sps_lmcs_enabled_flag )   sps_cs_explicit_flag u(1) rbsp_trailing_bits( ) }

Referring to Table 53, the SPS may include an explicit signaling enabledflag sps_cs_explicit_flag that shows (indicates) whether explicit chromaresidual scaling factor signaling or implicit chroma residual scalingfactor derivation is applied and/or an LMCS enabled flagsps_lmcs_enabled_flag showing whether the LMCS data field is enabled.The explicit signaling enabled flag may be included in the SPS on thebasis of the fact that the value of the LMCS enabled flag is 1.

The following table shows exemplary syntaxes of a slice header accordingto the present embodiment.

TABLE 54 slice_header( ) { Descriptor  slice_pic_parameter_set_id ue(v)...  if( sps_lmcs_enabled_flag ) {   slice_lmcs_enabled_flag u(1)   if(slice_lmcs_enabled_flag ) {    slice_lmcs_aps_id u(5)   slice_chroma_residual_scale_flag u(1)    if ( sps_cs_explicit_flag &&   slice_chroma_residual_scale_flag ) {     slice_cs_horiz_size_minus1ue(v)     slice_cs_vertic_size_minus1 ue(v)    }   }  } ... byte_alignment( ) }

Referring to FIG. 54, header information may include an LMCS enabledflag slice_lmcs_enabled_flag showing whether the LMCS data field isenabled at the current slice (or current picture). The LMCS enabled flagincluded in the SPS may be referred to as a first LMCS enabled flag andthe LMCS enabled flag included in the header information may be referredto as a second LMCS enabled flag. For example, when the value of thefirst LMCS enabled flag is 1, the second LMCS enabled flag may beincluded in the header information. When the value of the second LMCSenabled flag is 1, the header information may include LMCS APS relatedID information slice_lmcs_aps_id showing the ID of an LMCS APS, whichthe current picture refers to, and a chroma residual scaling enabledflag slice_chroma_residual_scale_flag. The header information may bepicture header information. That is, the syntaxes in Table 54 may beapplied to the information about a picture head, and in this case, theprefix slice_may be replaced with ph_.

In one example, the header information may include conditions based onan explicit signaling enabled flag sps_cs_explicit_flag and/or a chromaresidual scaling enabled flag slice_chroma_residual_scale_flag. When acondition that the explicit signaling enabled flag and the chromaresidual scaling enabled flag are both true (or 1) is true, the headerinformation may include information (ex.slice_cs_horiz_size_minus1,slice_cs_vertic_size_minus1) about the number of blocks sharing the samechroma residual scaling index.

The following table shows semantics about syntax elements included inTable 54.

TABLE 55 slice_cs_horiz_size_minus1 specifies the horizontal number ofCTUs sharing the same chroma residual scaling index. Whenslice_cs_horiz_size_minus1 is not present, it is inferred to be equal to0. The variable CsHorizSize is derived as ( slice_cs_horiz_size_minus1 +1 ). slice_cs_vertic_size_minus1 specifies the vertical number of CTUssharing the same chroma residual scaling index. Whenslice_cs_vertic_size_minus1 is not present, it is inferred to be equalto 0. The variable CsVerticSize is derived as (slice_cs_vertic_size_minus1 + 1 ).

Referring to Table 55, the information about the number of blockssharing the same chroma residual scaling index may include informationabout the horizontal number of blocks sharing the same chroma residualscaling index slice_cs_horiz_size_minus1 and/or information about thevertical number of blocks sharing the same chroma residual scaling indexslice_cs_vertic_size_minus1. The horizontal number CsVerticSize ofblocks sharing the same chroma residual scaling index may be derived onthe basis of the syntax element slice_cs_horiz_size_minus1, and thevertical number CsVerticSize of blocks sharing the same chroma residualscaling index may be derived on the basis of the syntax elementslice_cs_vertic_size_minus1. In this case, the horizontal number ofblocks may mean the horizontal length of the region to which the blockssharing the same chroma residual scaling index pertain, and the verticalnumber of blocks may mean the vertical length of the region to which theblocks sharing the same chroma residual scaling index pertain. In oneexample, the region to which the blocks sharing the same chroma residualscaling index pertain may be the product of the horizontal number andthe vertical number.

The following table shows exemplary syntaxes of coding (tree) unitinformation according to the present embodiment.

TABLE 56 coding_tree_unit( ) { Descriptor    xCtb − ( CtbAddrInRs %PicWidthInCtbsY ) <<  CtbLog2SizeY  yCtb − ( CtbAddrInRs /PicWidthInCtbsY ) <<  CtbLog2SizeY ...  if( sps_cs_explicit_flag && slice_chroma_residual_scale_flag ) {   if ( ( xCtb >> CtbLog2SizeY ) %   CsHorizSize − − 0 && ( yCtb >>    CtbLog2SizeY ) % CsVerticSize = = 0) {    cu_crs_delta_abs ae(v)    if ( cu_crs_delta_abs )    cu_crs_delta_sign_flag ae(v)   }  }  if( sps_cs_explicit_flag && slice_chroma_residual_scale_flag ) {  }  } ... }

Referring to Table 56, the coding unit information may includeconditions based on an explicit signaling enabled flagsps_cs_explicit_flag and/or a chroma residual scaling enabled flagslice_chroma_residual_scale_flag. When a condition that the explicitsignaling enabled flag and the chroma residual scaling enabled flag areboth true (or 1) is true, the coding unit information may includeinformation about the value(s) of a chroma residual scaling factor. Theinformation about the value(s) of a chroma residual scaling factor mayinclude information about the absolute value of a chroma residualscaling delta value cu_crs_delta_abs and/or informationcu_crs_delta_sign_flagabout the sign of the chroma residual scalingdelta value. For example, when the value of the information about theabsolute value of the chroma residual scaling delta value is 1 (or thevalue of the information about the absolute value of the chroma residualscaling delta value is larger than 0), the information about the sign ofthe chroma residual scaling delta value may be included in the codingunit information.

As a result, according to an embodiment of the present document,integrated chroma residual scaling may be applied to both a dual treearchitecture and a signal tree architecture. A high-level syntax (ex.SPS) may include enabled information about explicit signaling orimplicit derivation of a chroma residual scaling factor. The specialgranularity may be controlled from the CTU scale to the slice scalethrough explicit signaling. Implicit derivation may be applied to eachcurrent block using adjacent reconstructed luma samples included in thecurrent block VPDU. In accordance with the present embodiment, pipelinelatency in chroma residual scaling can be removed. Further, luma-chromadependency can be removed through explicit signaling. That is, theinformation about the value(s) of an explicit chroma residual scalingfactor may be explicitly signaled, and accordingly, dependency betweenluma samples and chroma samples may be removed.

The figures to be described below were constructed to describe adetailed example of the present document. The detailed names ofapparatuses or the detailed names of signals/messages/fields describedin the figures are provided only examples, and the technicalcharacteristics of the present specification are not limited to thedetailed names used in the following figures.

FIGS. 16 and 17 schematically show an example of a video/image encodingmethod according to (an) embodiment(s) of the present document andrelevant components. The method shown in FIG. 16 may be performed by theencoding device disclosed in FIG. 2. In detail, for example, in FIG. 16,S1600 and/or S1610 may be performed by the residual processor 230 of theencoding device, S1620 to S1640 may be performed by the predictor 220 ofthe encoding device, S1650 to S1680 may be performed by the residualprocessor 230 of the encoding device, and S1690 may be performed by theentropy encoder 240 of the encoding device. The method disclosed in FIG.16 may include the embodiments described above in the present document.

Referring to FIG. 16, an encoding device may derive LMCS codewords(S1600). The encoding device may derive LMCS codewords for a pluralityof bins, respectively. For example, lmcsCW[i] described above maycorrespond to the LMCS codewords derived by the encoding device.

The encoding device may generate LMCS related information (S1610). Theencoding device may generate LMCS related information on the basis ofthe LMCS codewords. The encoding device may derive an LMCS relatedparameter that may be applied for filtering the reconstructed samples,and may generate LMCS related information on the basis of LMCS relatedparameters. For example, the LMCS related information may includeinformation about the luma mapping described above (ex. forward mapping,inverse mapping, and linear mapping), information about chroma residualscaling, and/or indexes (ex. a maximum bin index and a minimum binindex) related to LMCS (or reshaping or reshaper), information about theabsolute value of a chroma residual scaling delta value, informationabout the sign of a chroma residual scaling delta value, and/orinformation about the number of blocks sharing the same chroma residualscaling index.

The encoding device may derive a prediction mode for the current blockin the current picture (S1620). The encoding device may derive at leastone of various modes disclosed in the present document of a predictionmode for the current block.

The encoding device may generate prediction related information on thebasis of the prediction mode (S1630). The prediction related informationmay include information about a prediction procedure that is performedfor the current block. For example, the prediction information mayinclude prediction mode/type information.

The encoding device may generate prediction chroma samples on the basisof the prediction mode (S1640). The encoding device may derive theresidual chroma samples on the basis of the original chroma samples andprediction chroma samples of the current block (S1650). For example, theencoding device may derive residual chroma samples on the basis of thedifferences between the prediction chroma samples and the originalchroma samples.

Further, the encoding device may generate prediction luma samples on thebasis of the prediction mode. In this case, various prediction methodsdisclosed in the present document such as inter prediction or intraprediction may be applied.

The encoding device may generate mapped prediction luma samples. Forexample, the encoding device may derive input values and mapping values(output values) of pivot points for luma mapping, and may generateprediction luma samples mapped on the basis of the input values and themapping values. In one example, the encoding device may derive a mappingindex idxY on the basis of a first prediction luma sample and maygenerate a prediction luma sample primarily mapped on the basis of theinput value and the mapping value of the pivot point corresponding tothe mapping index. In another example, linear mapping (linear shaping,linear LMCS) may be used, prediction luma samples mapped on the basis ofa forward mapping scaling factor derived from two pivot points may begenerated in linear mapping, and accordingly, the index derivationprocedure may be omitted due to the linear mapping.

The encoding device may generate residual luma samples on the basis ofthe mapped prediction luma samples. For example, the encoding device mayderive residual luma samples on the basis of the differences between themapped prediction luma samples and the original luma samples.

The encoding device may generate reconstructed luma samples. Theencoding device may generate reconstructed luma samples on the basis ofthe mapped prediction luma samples. In detail, the encoding device maysum up the residual luma samples with the mapped prediction luma samplesand may generated reconstructed luma samples on the basis of thesummation result.

The encoding device may generate reconstructed luma samples modified onthe basis of the bins for the luma mapping, the LMCS codewords, and thereconstructed luma sample. The encoding device may generate the modifiedreconstructed luma samples through an inverse mapping procedure for thereconstructed luma samples. For example, the encoding device may derivean inverse mapping index (ex. invYIdx) on the basis of reconstructedluma samples and/or mapping values (ex. LmcsPivot[i], i=lmcs_min_bin_idx. . . LmcsMaxBinIdx+1) designated for bin indexes, respectively, in theinverse mapping procedure. The encoding device may generatereconstructed luma samples modified on the basis of the mapping valueLmcsPivot[invYIdx] designated to the inverse mapping index.

The encoding device may derive a chroma residual scaling factor for lumadependent chroma residual scaling on the basis of the LMCS codewords andluma samples (1660). For example, the encoding device may derive chromaresidual scaling coefficients for a plurality of bins on the basis ofthe LMCS codewords. The encoding device may derive an inverse scaling(bin) index (or a piecewise function index, ex. invYIdx) on the basis ofthe reconstructed luma samples. The encoding device may derive a chromaresidual scaling coefficient indicated by the inverse scaling index asthe chroma residual scaling factor.

The encoding device may generate scaled residual chroma samples (S1670).In detail, the encoding device may derive a chroma residual scalingfactor and may generate residual chroma samples scaled on the basis of achroma residual scaling factor. In this case, chroma residual scaling atan encoding end may be referred to as forward chroma residual scaling.Accordingly, the chroma residual scaling factor derived by the encodingdevice may be referred to as a forward chroma residual scaling factor,and forward-scaled residual chroma samples may be generated.

The encoding device may derive residual information (1680). The encodingdevice may derive residual information on the basis of the scaledresidual chroma samples and/or the residual luma samples. The encodingdevice may derive transform coefficients on the basis of a transformprocedure for the scaled residual chroma samples and the residual lumasamples. For example, the transform procedure may include at least oneof DCT, DST, GBT, or CNT. The encoding device may derive quantizedtransform coefficients on the basis of a quantization procedure for thetransform coefficients. The quantized transform coefficients may have aone-dimensional vector form on the basis of a coefficient scan order.The encoding device may generate residual information showing thequantized transform coefficient. The residual information may begenerated through various encoding methods such as exponential Golomb,CAVLC, and CABAC.

The encoding device may encode image/video information (S1690). Theimage information may include LMCS related information and/or residualinformation. For example, the LMCS related information may includelinear LMCS related information. For example, at least one LMCS codewordmay be derived on the basis of linear LMCS related information. Theencoded video/image information may be output in the form of abitstream. The bitstream may be transmitted to a decoding device througha network of a storage medium.

The image/video information may include various items of informationaccording to an embodiment of the present document. For example, theimage/video information may include information disclosed in at leastany one of Tables 1 to 56 described above.

In one embodiment, the image information may include information aboutthe absolute value of a chroma residual scaling delta value (ex.cu_crs_delta_abs) and information about the sign of the chroma residualscaling delta value (ex. cu_crs_delta_sign_flag). For example, theinformation about the absolute value of a chroma residual scaling deltavalue and the information about the sign of the chroma residual scalingdelta value may be information for the chroma residual scaling factor.That is, the information about the absolute value of a chroma residualscaling delta value and the information about the sign of the chromaresidual scaling delta value may show the chroma residual scalingfactor.

In one embodiment, the image information may include an APS. The APS mayinclude an LMCS data field. The LMCS data field may include the LMCSrelated information.

In one embodiment, the image information may include an SPS. The SPS mayinclude a first LMCS enabled flag showing whether the LMCS data field isenabled.

In one embodiment, the image information may include header information.In this case, the header information may be information about a sliceheader or a picture header. On the basis of the fact that the value ofthe first LMCS enabled flag is 1, the header information may include asecond LCMS enabled flag showing whether the LMCS data field is enabledin a picture, and LMCS related APS ID information showing the ID of anLMCS APS that the current picture refers to.

In one embodiment, on the basis of the fact that he value of the secondLMCS enabled flag is 1, the LMCS related APS ID information may beincluded in the header information.

In one embodiment, when the current block is a single tree architectureor a dual tree architecture (when the current block is an individualtree architecture and the current block is coded in an individual tree,a chroma residual scaling enabled flag showing whether chroma residualscaling is applied for the current block may be included in the headerinformation (may be generated by the encoding device). When chromaresidual scaling is applied for the current picture, current slice,and/or current block, the value of the chroma residual scaling enabledflag may be 1.

In one embodiment, when the values of the first and second LMCS enabledflags are 1, the chroma residual scaling enabled flag may be included inthe header information.

In one embodiment, the image information may include information(slice_cs_horiz_size_munus1 or slice_cs_vertic_size_munus1 of Table 54and/or Table 55) about the number of blocks sharing the same chromaresidual scaling index. The chroma residual scaling factor may bederived on the basis of the information about the number of blockssharing the same chroma residual scaling index.

In one embodiment, the information about the number of blocks sharingthe same chroma residual scaling index may include information about thehorizontal number of blocks sharing the same chroma residual scalingindex slice_cs_horiz_size_munus1 of Table 54 and/or Table 55 and/orinformation about the vertical number of blocks sharing the same chromaresidual scaling index slice_cs_vertic_size_munus1 of Table 54 and/orTable 55.

In one embodiment, the encoding device may generate a piecewise indexfor chroma residual scaling. The encoding device may derive a chromaresidual scaling factor on the basis of the piecewise index. Theencoding device may generate residual chroma samples scaled on the basisof the residual chroma samples and the chroma residual scaling factor.

In one embodiment, when the current block is a single tree architectureor a dual tree architecture (when the current block is an individualtree architecture and the current block is coded in an individual tree,a chroma residual scaling enabled flag showing whether chroma residualscaling is applied for the current block may be generated by theencoding device. When chroma residual scaling is applied for the currentpicture, current slice, and/or current block, the value of the chromaresidual scaling enabled flag may be 1.

In one embodiment, the chroma residual scaling factor may be a singlechroma residual scaling factor.

In one embodiment, the image information may include an SPS (sequenceparameter set). The SPS may include a linear LMCS enabled flag showingwhether a linear LMCS is enabled.

In one embodiment, a minimum bin index (ex. lmcs_min_bin_idx) and/or amaximum bin index (ex. LmcsMaxBinIdx) may be derived on the basis of theinformation about the LMCS data. A first mapping valueLmcsPivot[lmcs_min_bin_idx] may be derived on the basis of the minimumbin index. A second mapping value LmcsPivot[LmcsMaxBinIdx] orLmcsPivot[LmcsMaxBinIdx+1] may be derived on the basis of the maximumbin index. The values of the reconstructed luma samples (ex., lumaSamplein Table 36 or 37) may be in the range from the first mapping value tothe second mapping value. In one embodiment, the value of allreconstructed luma samples may be in the range from the first mappingvalue to the second mapping value. In another embodiment, the values ofsome samples of the reconstructed luma samples may be in the range fromthe first mapping value to the second mapping value.

In one embodiment, the information about the LMCS data may includeinformation about an LMCS data field and a linear LMCS. The informationabout the linear LMCS may be referred to as information about linearmapping. The LMCS data filed may include an linear LMCS flag showingwhether linear LMCS is applied. When the value of the linear LMCS flagis 1, the mapped prediction luma samples may be generated on the basisof the information about the linear LMCS.

In one embodiment, the information about the linear LMCS may includeinformation about the first pivot point (ex. P1 in FIG. 11) and theinformation about the second pivot point (ex. P2 in FIG. 11). Forexample, the input value and the mapping value of the first pivot pointmay be a minimum input value and a minimum mapping value, respectively.The input value and the mapping value of the second pivot point may be amaximum input value and a maximum mapping value, respectively. An inputvalue between the minimum input value and the maximum input value may belinearly mapped.

In one embodiment, the image information may include information aboutthe maximum input value and information about the maximum mapping value.The maximum input value may be the same as the value of the informationabout the maximum input value (ex. lmcs_max_input in Table 33). Themaximum mapping value may be the same as the value of the informationabout the maximum mapping value (ex. lmcs_max_mapped in Table 33).

In one embodiment, the information about linear mapping may includeinformation about the input delta value of the second pivot point (ex.lmcs_max_input_delta in Table 35) and information about the mappingdelta value of the second pivot point (ex. lmcs_max_mapped_delta inTable 35). The maximum input value may be derived on the basis of theinput delta value of the second pivot point and the maximum mappingvalue may be derived on the basis of the mapping delta value of thesecond pivot point.

In one embodiment the maximum input value and the maximum mapping valuemay be derived on at least one equation included in Table 36 describedabove.

In one embodiment, a step of generating the mapped prediction lumasamples may include a step of deriving a forward mapping scaling factor(ex. ScaleCoeffSingle described above) for the prediction luma samples,and a step of generating the mapped prediction luma samples on the basisof the forward mapping scaling factor. The forward mapping scalingfactor may be a single factor for the prediction luma samples.

In one embodiment, the forward mapping scaling factor may be derived onthe basis of at least one equation included in Table 36 and/or Table 38described above.

In one embodiment, the mapped prediction luma samples may be derived onthe basis of at least one equation included in Table 38 described above.

In one embodiment, the encoding device may derive an inverse mappingscaling factor (ex. InvScaleCoeffSingle described above) for thereconstructed luma samples (ex. lumaSample described above). Further,the encoding device may generate reconstructed luma samples (ex.invSample) modified on the basis of the reconstructed samples and theinverse mapping scaling factor. The inverse mapping scaling factor maybe a single factor for the reconstructed luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedusing a piecewise index derived on the basis of the reconstructed lumasamples.

In one embodiment, the piecewise index may be derived on Table 50described above. That is, the comparison procedurelumaSample<LmcsPivot[idxYInv+1] included in Table 50 may be repeatedlyperformed from when the piecewise index is the minimum bin index to whenthe piecewise index is the maximum bin index.

In one embodiment, the inverse mapping scaling factor may be derived onthe basis of at least one equation included in Tables 33, 34, 35, and 36described above, or Equation 11 or 12.

In one embodiment, the modified reconstructed luma samples may bederived on the basis of Equation 20, Equation 21, Table 39, and/or Table40.

In one embodiment, the LMCS related information may include informationabout the number of bins for deriving the mapped prediction luma samples(ex. lmcs_num_bins_minus1 of Table 41). For example, the number of pivotpoints for luma mapping may be set the same as the number of the bins.In one example, the encoding device may generate delta input values anddelta mapping values of the pivot points by the number of the bins,respectively. In one example, the input values and mapping values of thepivot points are derived on the basis of the delta input values (ex.lmcs_delta input_cw[i] of Table 41) and the delta mapping values (ex.lmcs_delta_mapped_cw[i] of Table 41), and the mapped prediction lumasamples may be generated on the basis of the input values (ex.LmcsPivot_input[i] of Table 42) and the mapping values (ex.LmcsPivot_mapped[i] of Table 42).

In one embodiment, the encoding device may derive an LMCS delta codewordon the basis of at least one LMCS codeword included in the LMCS relatedinformation and the original codeword OrgCW, and may derive lumaprediction samples mapped on the basis of the at least one LMCS codewordand the original codeword. In one example, the information about linearmapping may include information about an LMCS delta codeword.

In one embodiment, the at least one LMCS codeword may be derived on thebasis of summation of the LMCS delta codeword and the OrgCW, and forexample, the OrgCW may be (1<<BitDepthY)/16, in which BitDepthY may showa luma bit depth. The present embodiment may be based on Equation 12.

In one embodiment, the at least one LMCS codeword may be derived on thebasis of summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx-lmcs_min_bin_idx+1), lmcs_max_bin_idx andlmcs_min_bin_idx may be a maximum bin index and a minimum bin index,respectively, and OrgCW may be (1<<BitDepthY)/16. The present embodimentmay be based on Equations 15 and 16.

In one embodiment, the at least one LMCS codeword may be a multiple of2.

In one embodiment, when the luma bit depth BitDepthY of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY-10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

FIGS. 18 and 19 schematically show an example of a image/video decodingmethod according to an embodiment of the present document and relevantcomponents. The method shown in FIG. 18 may be performed by the decodingdevice disclosed in FIG. 3. In detail, for example, in FIG. 18, S1800may be performed by the entropy decoder 310 of the decoding device,S1810 may be performed by the residual processor 320, the predictor 330,and/or the adder 340 of the decoding device, S1820 may be performed bythe predictor 330 of the decoding device, S1830 may be performed by theresidual processor 320 of the decoding device, S1840 may be performed bythe residual processor 320, the predictor 330, and/or the adder 340 ofthe decoding device, and S1850 may be performed by the adder 340 of thedecoding device. The method disclosed in FIG. 18 may include theembodiments described above in the present document.

Referring to FIG. 18, the decoding device may receive/obtain video/imageinformation (S1800). The video/image information may include LMCSrelated information, prediction related information, and/or residualinformation. For example, the LMCS related information may includeinformation about the luma mapping described above (ex. forward mapping,inverse mapping, and linear mapping), information about chroma residualscaling, indexes (ex. a maximum bin index, a minimum bin index, and amapping index) related to LMCS (or reshaping or reshaper), informationabout the absolute value of a chroma residual scaling delta value,information about the sign of a chroma residual scaling delta value,and/or information about the number of blocks sharing the same chromaresidual scaling index. The decoding device may receive/obtain theimage/video information through a bitstream.

The image/video information may include various items of informationaccording to an embodiment of the present document. For example, theimage/video information may include information disclosed in at leastany one of Tables 1 to 56 described above.

The decoding device may derive LMCS codewords on the basis of the LMCSrelated information (S1810). The decoding device may derive informationabout a plurality of bins and information about LMCS codewords on thebasis of the LMCS related information. The decoding device may deriveLMCS codewords for a plurality of bins, respectively. For example,lmcsCW[i] described above may correspond to the LMCS codewords derivedon the basis of the LMCS related information.

The decoding device may generate prediction luma samples. The decodingdevice may derive a prediction mode for the current block on the basisof prediction related information and may derive prediction luma samplesof the current block on the basis of the prediction mode. In this case,various prediction methods disclosed in the present document such asinter prediction or intra prediction may be applied.

The decoding device may generate prediction chroma samples (S1820). Thedecoding device may derive a prediction mode for the current block onthe basis of prediction related information and may derive predictionchroma samples of the current block on the basis of the prediction mode.In this case, various prediction methods disclosed in the presentdocument such as inter prediction or intra prediction may be applied.

The decoding device may generate residual chroma samples on the basis ofthe residual information (S1830). Further, the decoding device maygenerate residual luma samples on the basis of the residual information.In detail, the decoding device may derive transform coefficientsquantized on the basis of residual information. The quantized transformcoefficients may have a one-dimensional vector form on the basis of acoefficient scan order. The decoding device may derive transformcoefficients on the basis of a dequantization procedure for thequantized transform coefficients. The decoding device may deriveresidual chroma samples and/or residual luma samples on the basis oftransform coefficients.

The decoding device may generate mapped prediction luma samples. Forexample, the decoding device may derive input values and mapping values(output values) of pivot points for luma mapping, and may generateprediction luma samples mapped on the basis of the input values and themapping values. In one example, the decoding device may derive a(forward) mapping index idxY on the basis of a first prediction lumasample and may generate a prediction luma sample primarily mapped on thebasis of the input value and the mapping value of the pivot pointcorresponding to the mapping index. In another example, linear mapping(linear shaping, linear LMCS) may be used, prediction luma samplesmapped on the basis of a forward mapping scaling factor derived from twopivot points may be generated in linear mapping, and accordingly, theindex derivation procedure may be omitted due to the linear mapping.

The decoding device may generate reconstructed luma samples. Thedecoding device may generate reconstructed luma samples on the basis ofthe mapped prediction luma samples. In detail, the decoding device maysum up the residual luma samples with the mapped prediction luma samplesand may generated reconstructed luma samples on the basis of thesummation result.

The decoding device may derive a chroma residual scaling factor for lumadependent chroma residual scaling on the basis of the LMCS codewords andluma samples (S1840). For example, the decoding device may derive chromaresidual scaling coefficients (ChromaScaleCoeff[i] included in Table 19and/or Table 20) for a plurality of bins. The decoding device may derivean inverse scaling (bin) index (or a piecewise function index) on thebasis of the reconstructed luma samples. The decoding device may derivethe chroma residual scaling factor on the basis of the chroma residualscaling coefficients for the plurality of bins and the inverse scalingindex. In one example, the decoding device derive a chroma residualscaling coefficient indicated by the inverse scaling bin index of thechroma residual scaling coefficients for the plurality of bins as thechroma residual scaling factor. Derivation of an inverse scaling indexmay be achieved on the basis of Table 51 and/or Table 52 describedabove.

The decoding device may generate scaled residual chroma samples. Thedecoding device may generate residual chroma samples scaled on the basisof the residual chroma samples and the chroma residual scaling factor.For example, the decoding device may generate the scaled residual chromasamples on the basis of the product of (the absolute values of) theresidual chroma samples and the chroma residual scaling factor. In thiscase, chroma residual scaling at a decoding end may be, opposite to theencoding end, referred to as inverse chroma residual scaling.Accordingly, the chroma residual scaling factor derived by the decodingdevice may be referred to as an inverse chroma residual scaling factor,and inverse-scaled residual chroma samples may be generated.

The decoding device may generate reconstructed chroma samples (S1850).The decoding device may generate reconstructed chroma samples on thebasis of the prediction chroma samples, the residual chroma samples,and/or the chroma residual scaling factor. In detail, the decodingdevice may generate the reconstructed chroma samples by summing up thescaled residual chroma samples with prediction chroma samples.

The decoding device may generate modified reconstructed luma samples onthe basis of the LMCS codewords and the reconstructed luma samples. Thedecoding device may generate the modified reconstructed luma samplesthrough an inverse mapping procedure for the reconstructed luma samples.

In one embodiment, the image information may include information aboutthe absolute value of a chroma residual scaling delta value (ex.cu_crs_delta_abs) and information about the sign of the chroma residualscaling delta value (ex. cu_crs_delta_sign_flag). For example, theinformation about the absolute value of a chroma residual scaling deltavalue and the information about the sign of the chroma residual scalingdelta value may be information for the chroma residual scaling factor.That is, the chroma residual scaling factor may be derived on the basisof the information about the absolute value of a chroma residual scalingdelta value and the information about the sign of the chroma residualscaling delta value.

In one embodiment, the image information may include an APS. Forexample, the APS may include an LMCS data field. The LMCS data field mayinclude the LMCS related information.

In one embodiment, the image information may include an SPS. The SPS mayinclude a first LMCS enabled flag showing whether the LMCS data field isenabled.

In one embodiment, the image information may include header information.In this case, the header information may be information about a sliceheader or a picture header. The header information may include a secondLMCS enabled flag showing whether the LMCS data filed is enabled in apicture on the basis of the fact that the value of the first LMCSenabled flag is 1.

In one embodiment, on the basis of the fact that he value of the secondLMCS enabled flag is 1, the LMCS related APS ID information may beincluded in the header information.

In one embodiment, when the current block is a single tree architectureor a dual tree architecture (when the current block is an individualtree architecture and the current block is coded in an individual tree,a chroma residual scaling enabled flag showing whether chroma residualscaling is applied for the current block may be signaled. When chromaresidual scaling is applied for the current picture, current slice,and/or current block, the value of the chroma residual scaling enabledflag may be 1.

In one embodiment, when the values of the first and second LMCS enabledflags are 1, the chroma residual scaling enabled flag may be included inthe header information.

In one embodiment, the image information may include information(slice_cs_horiz_size_munus1 or slice_cs_vertic_size_munus1 of Table 54and/or Table 55) about the number of blocks sharing the same chromaresidual scaling index. The chroma residual scaling factor may bederived on the basis of the information about the number of blockssharing the same chroma residual scaling index.

In one embodiment, the information about the number of blocks sharingthe same chroma residual scaling index may include information about thehorizontal number of blocks sharing the same chroma residual scalingindex slice_cs_horiz_size_munus1 of Table 54 and/or Table 55 and/orinformation about the vertical number of blocks sharing the same chromaresidual scaling index slice_cs_vertic_size_munus1 of Table 54 and/orTable 55.

In one embodiment, a piecewise index (ex. idxYInv of Table 35, 36, or37) may be identified on the basis of the information about LMCS data.The decoding device may derive a chroma residual scaling factor on thebasis of the piecewise index. The decoding device may generate residualchroma samples scaled on the basis of the residual chroma samples andthe chroma residual scaling factor.

In one embodiment, when the current block is a single tree architectureor a dual tree architecture (when the current block is an individualtree architecture and the current block is coded in an individual tree,a chroma residual scaling enabled flag showing whether chroma residualscaling is applied for the current block may be signaled. When chromaresidual scaling is applied for the current picture, current slice,and/or current block, the value of the chroma residual scaling enabledflag may be 1.

In one embodiment, the chroma residual scaling factor may be a singlechroma residual scaling factor.

In one embodiment, the APS may include the LMCS data field includingLMCS parameters on the basis of the fact that the value of the typeinformation is 1.

In one embodiment, the image information may include header information.In this case, the header information may be a picture header (or a sliceheader). The header information may include LMCS related APS IDinformation. The LMCS related APS ID information may show the identifierinformation of an LMCS APS for the current picture, the current block,or a current slice. That is, the value of the LMCS related APS IDinformation may be the same as the value of the identifier informationof the LMCS APS. For example, the value of the LMCS related APS IDinformation may be within the range from 0 to 3. That is, the value ofthe LMCS related APS ID information may be 0, 1, 2, or 3.

In one embodiment, a minimum bin index (ex. lmcs_min_bin_idx) and/or amaximum bin index (ex. LmcsMaxBinIdx) may be derived on the basis of theinformation about the LMCS data. A first mapping valueLmcsPivot[lmcs_min_bin_idx] may be derived on the basis of the minimumbin index. A second mapping value LmcsPivot[LmcsMaxBinIdx] orLmcsPivot[LmcsMaxBinIdx+1] may be derived on the basis of the maximumbin index. The values of the reconstructed luma samples (ex., lumaSamplein Table 51 or 52) may be in the range from the first mapping value tothe second mapping value. In one embodiment, the value of allreconstructed luma samples may be in the range from the first mappingvalue to the second mapping value. In another embodiment, the values ofsome samples of the reconstructed luma samples may be in the range fromthe first mapping value to the second mapping value.

In one embodiment, the image information may include an SPS (sequenceparameter set). The SPS may include a linear LMCS enabled flag showingwhether a linear LMCS is enabled.

In one embodiment, the chroma residual scaling factor may be a singlechroma residual scaling factor.

In one embodiment, the information about the LMCS data may includeinformation about an LMCS data field and a linear LMCS. The informationabout the linear LMCS may be referred to as information about linearmapping. The LMCS data filed may include an linear LMCS flag showingwhether linear LMCS is applied. When the value of the linear LMCS flagis 1, the mapped prediction luma samples may be generated on the basisof the information about the linear LMCS.

In one embodiment, the information about the linear LMCS may includeinformation about the first pivot point (ex. P1 in FIG. 11) and theinformation about the second pivot point (ex. P2 in FIG. 11). Forexample, the input value and the mapping value of the first pivot pointmay be a minimum input value and a minimum mapping value, respectively.The input value and the mapping value of the second pivot point may be amaximum input value and a maximum mapping value, respectively. An inputvalue between the minimum input value and the maximum input value may belinearly mapped.

In an embodiment, the image information may include information aboutthe maximum input value and information about the maximum mapping value.The maximum input value may be the same as a value of information aboutthe maximum input value (e.g., lmcs_max_input in Table 33). The maximummapping value may be the same as a value of information about themaximum mapping value (e.g., lmcs_max_mapped in Table 33).

In an embodiment, the information on the linear mapping may includeinformation on the input delta value of the second pivot point (e.g.,lmcs_max_input_delta of Table 35) and information on the mapping deltavalue of the second pivot point (e.g., lmcs_max_mapped_delta of Table35). The maximum input value may be derived based on the input deltavalue of the second pivot point, and the maximum mapping value may bederived based on the mapping delta value of the second pivot point.

In an embodiment, the maximum input value and the maximum mapping valuemay be derived based on at least one equation included in Table 36described above.

In one embodiment, generating the mapped prediction luma samplescomprises deriving a forward mapping scaling factor (i.e.,ScaleCoeffSingle) for the prediction luma samples, and generating themapped prediction luma samples based on the forward mapping scalingfactor. The forward mapping scaling factor may be a single factor forthe prediction luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedusing a piecewise index derived based on the reconstructed luma samples.

In one embodiment, the piecewise index may be derived based on Table 51described above. That is, the comparison process included in Table 51(lumaSample<LmcsPivot[idxYInv+1]) may be iteratively performed from thepiecewise index being the minimum bin index to the piecewise index beingthe maximum bin index.

In one embodiment, the forward mapping scaling factor may be derivedbased on at least one equation included in Tables 36 and/or 38 describedabove.

In one embodiment, the mapped prediction luma samples may be derivedbased on at least one equation included in Table 38 described above.

In one embodiment, the decoding apparatus may derive an inverse mappingscaling factor (i.e., InvScaleCoeffSingle) for the reconstructed lumasamples (i.e., lumaSample). Also, the decoding apparatus may generatemodified reconstructed luma samples (i.e., invSample) based on thereconstructed luma samples and the inverse mapping scaling factor. Theinverse mapping scaling factor may be a single factor for thereconstructed luma samples.

In one embodiment, the inverse mapping scaling factor may be derivedbased on at least one of equations included in Tables 33, 34, 35, and36, or Equation 11 or 12 described above.

In one embodiment, the modified reconstructed luma samples may bederived based on Equation 20, Equation 21, Table 39, and/or Table 40described above.

In one embodiment, the LMCS related information may include informationon the number of bins for deriving the mapped prediction luma samples(i.e., lmcs_num_bins_minus1 in Table 41). For example, the number ofpivot points for luma mapping may be set equal to the number of bins. Inone example, the decoding apparatus may generate the delta input valuesand delta mapped values of the pivot points by the number of bins,respectively. In one example, the input values and mapped values of thepivot points are derived based on the delta input values (i.e.,lmcs_delta_input_cw[i] in Table 41) and the delta mapped values (i.e.,lmcs_delta_mapped_cw[i] in Table 41), and the mapped prediction lumasamples may be generated based on the input values (i.e.,LmcsPivot_input[i] of Table 42) and the mapped values (i.e.,LmcsPivot_mapped[i] of Table 42).

In one embodiment, the decoding apparatus may derive an LMCS deltacodeword based on at least one LMCS codeword and an original codeword(OrgCW) included in the LMCS related information, and mapped lumaprediction samples may be derived based on the at least one LMCScodeword and the original codeword. In one example, the information onthe linear mapping may include information on the LMCS delta codeword.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword and the OrgCW, for example,OrgCW is (1<<BitDepthY)/16, where BitDepthY represents a luma bit depth.This embodiment may be based on Equation 14.

In one embodiment, the at least one LMCS codeword may be derived basedon the summation of the LMCS delta codeword andOrgCW*(lmcs_max_bin_idx-lmcs_min_bin_idx+1), for example,lmcs_max_bin_idx and lmcs_min_bin_idx are a maximum bin index and aminimum bin index, respectively, and OrgCW may be (1<<BitDepthY)/16.This embodiment may be based on Equations 15 and 16.

In one embodiment, the at least one LMCS codeword may be a multiple oftwo.

In one embodiment, when the luma bit depth (BitDepthY) of thereconstructed luma samples is higher than 10, the at least one LMCScodeword may be a multiple of 1<<(BitDepthY-10).

In one embodiment, the at least one LMCS codeword may be in the rangefrom (OrgCW>>1) to (OrgCW<<1)−1.

In the above-mentioned paragraphs, the information on the LMCS data maybe the same as the information on the LMCS.

In the above-described embodiment, the methods are described based onthe flowchart having a series of steps or blocks. The present disclosureis not limited to the order of the above steps or blocks. Some steps orblocks may occur simultaneously or in a different order from other stepsor blocks as described above. Further, those skilled in the art willunderstand that the steps shown in the above flowchart are notexclusive, that further steps may be included, or that one or more stepsin the flowchart may be deleted without affecting the scope of thepresent disclosure.

The method according to the above-described embodiments of the presentdocument may be implemented in software form, and the encoding deviceand/or decoding device according to the present document is, forexample, may be included in the device that performs the imageprocessing of a TV, a computer, a smart phone, a set-top box, a displaydevice, etc.

When the embodiments in the present document are implemented insoftware, the above-described method may be implemented as a module(process, function, etc.) that performs the above-described function. Amodule may be stored in a memory and executed by a processor. The memorymay be internal or external to the processor, and may be coupled to theprocessor by various well-known means. The processor may include anapplication-specific integrated circuit (ASIC), other chipsets, logiccircuits, and/or data processing devices. Memory may include read-onlymemory (ROM), random access memory (RAM), flash memory, memory cards,storage media, and/or other storage devices. That is, the embodimentsdescribed in the present document may be implemented and performed on aprocessor, a microprocessor, a controller, or a chip. For example, thefunctional units shown in each figure may be implemented and performedon a computer, a processor, a microprocessor, a controller, or a chip.In this case, information on instructions or an algorithm forimplementation may be stored in a digital storage medium.

In addition, the decoding apparatus and the encoding apparatus to whichthe present disclosure is applied may be included in a multimediabroadcasting transmission/reception apparatus, a mobile communicationterminal, a home cinema video apparatus, a digital cinema videoapparatus, a surveillance camera, a video chatting apparatus, areal-time communication apparatus such as video communication, a mobilestreaming apparatus, a storage medium, a camcorder, a VoD serviceproviding apparatus, an Over the top (OTT) video apparatus, an Internetstreaming service providing apparatus, a three-dimensional (3D) videoapparatus, a teleconference video apparatus, a transportation userequipment (i.e., vehicle user equipment, an airplane user equipment, aship user equipment, etc.) and a medical video apparatus and may be usedto process video signals and data signals. For example, the Over the top(OTT) video apparatus may include a game console, a blue-ray player, aninternet access TV, a home theater system, a smart phone, a tablet PC, aDigital Video Recorder (DVR), and the like.

Furthermore, the processing method to which the present document isapplied may be produced in the form of a program that is to be executedby a computer and may be stored in a computer-readable recording medium.Multimedia data having a data structure according to the presentdisclosure may also be stored in computer-readable recording media. Thecomputer-readable recording media include all types of storage devicesin which data readable by a computer system is stored. Thecomputer-readable recording media may include a BD, a Universal SerialBus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, a magnetic tape, afloppy disk, and an optical data storage device, for example.Furthermore, the computer-readable recording media includes mediaimplemented in the form of carrier waves (i.e., transmission through theInternet). In addition, a bitstream generated by the encoding method maybe stored in a computer-readable recording medium or may be transmittedover wired/wireless communication networks.

In addition, the embodiments of the present document may be implementedwith a computer program product according to program codes, and theprogram codes may be performed in a computer by the embodiments of thepresent document. The program codes may be stored on a carrier which isreadable by a computer.

FIG. 20 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

Referring to FIG. 20, the content streaming system to which theembodiment(s) of the present document is applied may largely include anencoding server, a streaming server, a web server, a media storage, auser device, and a multimedia input device.

The encoding server compresses content input from multimedia inputdevices such as a smartphone, a camera, a camcorder, etc. Into digitaldata to generate a bitstream and transmit the bitstream to the streamingserver. As another example, when the multimedia input devices such assmartphones, cameras, camcorders, etc. directly generate a bitstream,the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstreamgenerating method to which the embodiment(s) of the present disclosureis applied, and the streaming server may temporarily store the bitstreamin the process of transmitting or receiving the bitstream.

The streaming server transmits the multimedia data to the user devicebased on a user's request through the web server, and the web serverserves as a medium for informing the user of a service. When the userrequests a desired service from the web server, the web server deliversit to a streaming server, and the streaming server transmits multimediadata to the user. In this case, the content streaming system may includea separate control server. In this case, the control server serves tocontrol a command/response between devices in the content streamingsystem.

The streaming server may receive content from a media storage and/or anencoding server. For example, when the content is received from theencoding server, the content may be received in real time. In this case,in order to provide a smooth streaming service, the streaming server maystore the bitstream for a predetermined time.

Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcasting terminal, a personal digitalassistant (PDA), a portable multimedia player (PMP), navigation, a slatePC, tablet PCs, ultrabooks, wearable devices (ex. Smartwatches, smartglasses, head mounted displays), digital TVs, desktops computer, digitalsignage, and the like. Each server in the content streaming system maybe operated as a distributed server, in which case data received fromeach server may be distributed.

Each server in the content streaming system may be operated as adistributed server, and in this case, data received from each server maybe distributed and processed.

The claims described herein may be combined in various ways. Forexample, the technical features of the method claims of the presentdocument may be combined and implemented as an apparatus, and thetechnical features of the apparatus claims of the present document maybe combined and implemented as a method. In addition, the technicalfeatures of the method claim of the present document and the technicalfeatures of the apparatus claim may be combined to be implemented as anapparatus, and the technical features of the method claim of the presentdocument and the technical features of the apparatus claim may becombined and implemented as a method.

1. An image decoding method that is performed by a decoding device, theimage decoding method comprising: a step of obtaining image informationincluding LMCS (luma mapping with chroma scaling) related information,prediction related information, and residual information from abitstream; a step of deriving LMCS codewords on the basis of the LMCSrelated information; a step of generating prediction chroma samples onthe basis of the prediction related information; a step of generatingresidual chroma samples on the basis of the residual information; a stepof deriving a chroma residual scaling factor for luma dependent chromaresidual scaling on the basis of the LMCS codewords and luma samples;and a step of generating reconstructed chroma samples on the basis ofthe chroma residual scaling factor, wherein the image informationincludes information about the absolute value of a chroma residualscaling delta value and information about the sign of a chroma residualscaling delta value, and the chroma residual scaling factor is derivedon the basis of the information about the absolute value of a chromaresidual scaling delta value and the information about the sign of thechroma residual scaling delta value.
 2. The image decoding method ofclaim 1, wherein the image information includes an APS (adaptationparameter set), the APS includes an LMCS data field, and the LMCS datafield includes the LMCS related information.
 3. The image decodingmethod of claim 2, wherein the image information includes an SPS(sequence parameter set), and the SPS includes a first LMCS enabled flagshowing whether the LMCS data field is enabled.
 4. The image decodingmethod of claim 3, wherein the image information includes headerinformation, and the header information includes a second LCMS enabledflag showing whether the LMCS data field is enabled in a currentpicture, and LMCS related APS ID information showing the ID of LMCS APSthat the current picture refers to, on the basis of the fact that thevalue of the first LMCS enabled flag is
 1. 5. The image decoding methodof claim 3, wherein when the current block is a single tree architectureor a dual tree architecture, a chroma residual scaling enabled flagshowing whether chroma residual scaling is applied for the current blockis included in the header information.
 6. The image decoding method ofclaim 5, wherein when the values of the first and second LMCS enabledflag are 1, the chroma residual scaling enabled flag is included in theheader information.
 7. The image decoding method of claim 3, wherein theimage information includes information about the number of blockssharing a same chroma residual scaling index, and the chroma residualscaling factor is derived on the basis of the information about thenumber of blocks sharing the same chroma residual scaling index.
 8. Theimage decoding method of claim 7, wherein the information about thenumber of blocks sharing the same chroma residual scaling index includesinformation about a horizontal number of the blocks sharing the samechroma residual scaling index and information about a vertical number ofthe blocks sharing the same chroma residual scaling index.
 9. An imageencoding method that is performed by an encoding device, the imageencoding method comprising: a step of deriving LMCS (luma mapping withchroma scaling) codewords; a step of generating LMCS related informationon the basis of the LMCS codewords; a step of deriving a prediction modefor a current block for a current picture; a step of generatingprediction related information on the basis of the prediction mode; astep of generating prediction chroma samples on the basis of theprediction mode; a step of generating residual chroma samples on thebasis of the prediction chroma samples; a step of deriving a chromaresidual scaling factor for luma dependent chroma residual scaling onthe basis of the LMCS codewords and luma samples; a step of generatingresidual chroma samples scaled on the basis of the chroma residualscaling factor; a step of generating residual information on the basisof the scaled residual chroma samples; and a step of encoding imageinformation including the LMCS related information, prediction relatedinformation, and the residual information, wherein the image informationincludes information about the absolute value of a chroma residualscaling delta value and information about the sign of a chroma residualscaling delta value, and the information about the absolute value of achroma residual scaling delta value and the information about the signof the chroma residual scaling delta value show the chroma residualscaling factor.
 10. The image encoding method of claim 9, wherein theimage information includes an APS (adaptation parameter set), and theAPS includes an LMCS data field, and the LMCS data field includes theLMCS related information.
 11. The image encoding method of claim 10,wherein the image information includes an SPS (sequence parameter set),and the SPS includes a first LMCS enabled flag showing whether the LMCSdata field is enabled.
 12. The image encoding method of claim 11,wherein the image information includes header information, and theheader information includes a second LCMS enabled flag showing whetherthe LMCS data field is enabled in a current picture, and LMCS relatedAPS ID information showing the ID of LMCS APS that the current picturerefers to, on the basis of the fact that the value of the first LMCSenabled flag is
 1. 13. The image encoding method of claim 11, whereinwhen the current block is a single tree architecture or a dual treearchitecture, a chroma residual scaling enabled flag showing whetherchroma residual scaling is applied for the current block is included inthe header information.
 14. The image encoding method of claim 13,wherein when the values of the first and second LMCS enabled flag are 1,the chroma residual scaling enabled flag is included in the headerinformation.
 15. The image encoding method of claim 11, wherein theimage information includes information about the number of blockssharing a same chroma residual scaling index, and the chroma residualscaling factor is derived on the basis of the information about thenumber of blocks sharing the same chroma residual scaling index.
 16. Theimage encoding method of claim 15, wherein the information about thenumber of blocks sharing the same chroma residual scaling index includesinformation about a horizontal number of the blocks sharing the samechroma residual scaling index and information about a vertical number ofthe blocks sharing the same chroma residual scaling index.
 17. Anon-transitory computer-readable storage medium storing a bitstreamgenerated by a method, wherein the method comprising: a step of derivingLMCS (luma mapping with chroma scaling) codewords; a step of generatingLMCS related information on the basis of the LMCS codewords; a step ofderiving a prediction mode for a current block for a current picture; astep of generating prediction related information on the basis of theprediction mode; a step of generating prediction chroma samples on thebasis of the prediction mode; a step of generating residual chromasamples on the basis of the prediction chroma samples; a step ofderiving a chroma residual scaling factor for luma dependent chromaresidual scaling on the basis of the LMCS codewords and luma samples; astep of generating residual chroma samples scaled on the basis of thechroma residual scaling factor; a step of generating residualinformation on the basis of the scaled residual chroma samples; and astep of encoding image information including the LMCS relatedinformation, prediction related information, and the residualinformation to output the bitstream, wherein the image informationincludes information about the absolute value of a chroma residualscaling delta value and information about the sign of a chroma residualscaling delta value, and the information about the absolute value of achroma residual scaling delta value and the information about the signof the chroma residual scaling delta value show the chroma residualscaling factor.